When the Query Names Your Brand, Claude's Search Returns Your Own Site 33.3% of the Time
TL;DR: Across 150 runs of the Claude API's web search tool we logged what the search returned separately from what Claude cited. Where the query itself named a brand, that brand's own site was in the retrieved set in 36 of 108 run-brand pairs, 33.3%: 42.3% on comparison queries, 10% on alternatives. On how-to questions Claude searched in 1 of 24 runs. Two runs of the same query overlap 0.555.
Being cited by an AI assistant gets talked about as one event. It is two. A search has to run and it has to return your page. Then the model has to pick your page out of what came back. Those are two different steps with two different failure modes, and "we weren't cited" covers both without telling you which one happened.
We wanted to know how wide each step is on Claude. Calling the API's web search tool directly makes that measurable, because Claude's considered set comes back as structure: the response carries the search result blocks alongside the answer, so the retrieved list and the cited list arrive side by side on every request, already separated. (You can pull the same split off the browser surface: AirOps and Kevin Indig did exactly that, scraping ChatGPT's UI across 16,851 queries for The Fan-Out Effect. The API hands it to you instead of making you reconstruct it.)
One property of that format is worth stating before any of the numbers, because it shapes what the numbers can mean. A web search citation is not free text: each one carries an encrypted_index, which the API documents as "a reference that must be passed back" into the search results. The cited list is therefore a pointer into the retrieved list, a subset of it by construction. That is how the tool works rather than something we discovered, and the practical consequence is simply this: citation starts with getting retrieved.
This page reports The Claude Retrieval and Citation Study, round one: 50 ordinary B2B-SaaS buyer questions, three runs each, 150 requests against claude-sonnet-5 with the web search tool called directly, collected on 7 September 2026, logging what the search returned and what Claude cited on every single run.
So the first question we put to the data is the one a brand actually cares about. When the buyer's question has your name in it, does the search even return your site?
When the Query Names Your Brand, Claude's Search Returns Your Own Site 33.3% of the Time
- When the query names your brand, your own site is usually not in what the search returns
- The selection gate is wide: 62.7% of retrieved domains
- On how-to questions, Claude does not search at all
- Ask the same question twice and the cited lists overlap 0.555
- 458 domains retrieved, 333 cited, and a very long tail
- How this sits next to the published work
- What actually follows from this
- How we ran it
- What this round did not measure
- Cite this study
When the query names your brand, your own site is usually not in what the search returns
Twenty-three of the 50 queries name a brand in the query itself: phrasings like Notion vs Asana and Salesforce alternatives. There is no entity extraction step here and nothing to argue about. The brand is in the string the user typed. The only question is whether that brand's own site appeared in what the search handed back.
Mostly it did not.
| Run-brand pairs | Own site retrieved | |
|---|---|---|
| comparison | 78 | 33 (42.3%) |
| alternatives | 30 | 3 (10%) |
| Total | 108 | 36 (33.3%) |
In 108 chances to return the named brand's own website, the search returned it 36 times. On "alternatives" phrasings it happened three times out of thirty.
The brand-level picture is starker than the average, and it does not run the way most people would guess. Twelve of the 25 brands were never retrieved in a single run: Salesforce 0 of 9, Asana 0 of 6, HubSpot 0 of 6, Microsoft Teams 0 of 6, Zendesk 0 of 6, Zoom 0 of 6, plus DocuSign, Google Sheets, PayPal, Shopify, WooCommerce and Xero, all 0 of 3.
Seven brands were retrieved in every run they appeared in: ClickUp, Dropbox Sign, Gusto, Klaviyo, Monday and Rippling at 3 of 3 each, and QuickBooks at 6 of 6.
Read those two lists next to each other. The names that never came back are the largest ones in the sample. The names that always came back are mid-market. That is an observation, not an explanation, and we are not claiming to know the cause. The plausible mechanism is that a vendor's own page does not answer an "X vs Y" question and a comparison article does, so the retrieval step reaches for the article. We did not test that, and this round cannot separate it from any other explanation.
Membership of either list is itself unstable. When the 42 defective runs described below were re-run about twenty hours later, eight of the 25 brands changed their count against the identical queries and the identical matching rule. Slack went from retrieved in 1 of 6 runs to 3 of 6, Notion from 0 of 6 to 1 of 6, Mailchimp from 2 of 6 to 3 of 6, ClickUp, QuickBooks and Rippling each gained a run, and Zendesk fell from 1 of 6 to 0 of 6 while Freshdesk fell from 3 of 3 to 2 of 3. Two brands crossed a list boundary in the process: Notion left the never-retrieved list and Zendesk took its place. That is not a separate wrinkle in the measurement. It is the same run-to-run variance measured further down this page, showing up at brand level: which list a brand lands in is a reading taken on one day, not a standing property of the brand.
The matching rule moves this number more than the sample does, so here is the whole band. A brand with several properties could look "not retrieved" simply because we were watching the wrong hostname, so the table is computed against the full set of known domains for every brand and counts a subdomain of any of them: Freshdesk as freshdesk.com plus freshworks.com, WooCommerce as woo.com plus wordpress.org plus automattic.com, and sign.dropbox.com as Dropbox Sign, because it is. That is the rule this study's protocol states, and it gives 33.3%. Dropping subdomains gives 27.8%. Restricting each brand to one primary domain gives 30.6% with subdomains and 25.0% without. The published figure is the top of an 8.3 point band and it is the most generous reading of the four, which is deliberate: the finding should have to survive the rule that flatters the brands. It does, on all four. On most queries that name a brand, that brand's own site is not what the search returns.
The practical version is short. When a buyer asks about you by name, your own page is often not what the search hands the model. Being covered on the pages that do get retrieved is the lever you actually have.
The selection gate is wide: 62.7% of retrieved domains
Once a domain is in the retrieved set, being cited is the normal outcome rather than the lucky one.
| mean | median | range | |
|---|---|---|---|
| Domains retrieved per run | 11.98 | 12 | 6–21 |
| Domains cited per run | 7.31 | 7 | 4–12 |
| Share of retrieved that got cited | 62.7% | 62.5% | 35%–100% |
About twelve domains come back, about seven get cited. The unit here is the domain, not the page, and the 62.7% is the mean of the per-run ratios rather than an aggregate: pooling every domain slot across all runs gives 61.0%. Both are in the data and they answer slightly different questions, so the figure quoted throughout this page is the per-run mean. In some runs the share hit 100%: every domain the search returned ended up in the answer.
This is where you have to watch the unit, because the best-known numbers in this area are measured on a different object, and there are two separate AirOps reports that get quoted interchangeably when they should not be. In The Influence of Retrieval, Fan-out, and Google SERPs (12 March 2026, 43,233 queries), AirOps looked at 548,534 retrieved pages on ChatGPT and reported that it "left 85% of retrieved pages uncited". The later Fan-Out Effect report with Kevin Indig (13 April 2026, 16,851 queries, scraped from the interface) found that 58% of pages were never cited at all, and that citation rate fell from 58% at retrieval position 1 to 14% at position 10.
Neither of those contradicts our 62.7%. Different unit (pages against domains), different engine (ChatGPT against Claude), different collection method. They are answers to different questions and cannot be put on the same axis. Anywhere you see a retrieval-to-citation rate quoted, check what it is counting before you carry it anywhere.
What the two gates share is that they are wildly unequal in width, so the effort you spend on them should not be equal either. Selection discards a minority of what came back. Retrieval decides what "came back" is in the first place.
On how-to questions, Claude does not search at all
This is the most immediately usable result here, and it is the simplest.
| Intent | Runs where a search fired |
|---|---|
| recommendation | 57 of 57 (100%) |
| comparison | 39 of 39 (100%) |
| alternatives | 30 of 30 (100%) |
| how-to | 1 of 24 (4.2%) |
Recommendation, comparison and alternatives questions triggered a search in every run, without a single exception. How-to questions triggered one search in 24 runs. The search tool was optional on every request, with no tool_choice set, so this is Claude declining to search rather than us configuring it not to, and the same configuration produced 100% on the other three intents.
Seven of the eight how-to queries never searched in any of their three runs. The one exception was how to pick an email marketing platform, which searched in run 2 only.
The consequence is total rather than partial. An answer produced without a search has no search results, and a web search citation is a pointer into search results, so there is nothing for it to point at. On those questions there is no visibility contest. Not a hard one, not a crowded one. There is nothing to enter.
We are not the first to find this pattern, and we would rather say so than have you find out later. In January 2026, cloro (Ricardo Batista) published the ChatGPT Grounding Study: 100 prompts and 634 answers on ChatGPT, with the search rate split by intent. Commercial prompts triggered a search 86.5% of the time. Informational prompts, 0.9%. Across the six informational categories, 274 answers in total, not one search fired. The "how things work" category alone was 42 answers, 0.0%.
What this round adds is that the same pattern holds on a different engine, on B2B-SaaS buyer questions, split four ways by intent. It is not a ChatGPT quirk. And on Claude the commercial side is not 86.5% but complete: every one of the 57 recommendation runs, 39 comparison runs and 30 alternatives runs fired a search.
One published figure looks like a flat contradiction of this, and it is worth taking head on. The same AirOps retrieval study cited above split its 15,000 original prompts into eight intent types, 1,875 queries each, and how-to came out near the top: product-discovery and how-to queries "produced the highest citation rates at 18.3% and 16.9%", against 13.1% for comparison and 11.3% for validation. (That 15,000 is the original prompt count; the 43,233 quoted earlier on this page includes ChatGPT's own fan-out queries.) Read at speed, that says how-to is one of the better intents to compete on, which is the reverse of our table.
The denominators are not the same. AirOps is reporting the share of retrieved pages that ended up cited, a rate computed on queries where a search had already run and returned pages. Their question is: given retrieval, how likely is a page to be cited? Ours sits one step earlier: does a search run at all? On Claude, on how-to phrasings, it mostly did not, so there are no retrieved pages for a selection rate to be computed on. Both findings hold at once. If a how-to search fires, the pages it returns may well be cited at a good rate. In 23 of our 24 how-to runs, nothing fired.
The practical version: if the phrasing your buyer uses starts with "how to", a citation is not the prize available on that query. The phrasings that put sites in front of a reader in this study were the ones that named a category, named two products, or asked for alternatives.
Ask the same question twice and the cited lists overlap 0.555
Running every query three times lets us ask something a single-run study cannot: how stable is the cited list?
We measured overlap as Jaccard similarity between the cited domain sets of two runs of the same query, the intersection divided by the union, across the 42 queries that had at least two searching runs.
| Intent | Queries | Mean Jaccard |
|---|---|---|
| recommendation | 19 | 0.620 |
| alternatives | 10 | 0.587 |
| comparison | 13 | 0.435 |
| All | 42 | 0.555 (median 0.531) |
Two runs of the identical query, with nothing changed in the request, overlap at 0.555 on the cited domain set. Slightly more than half of the two runs' combined domains are common to both. The rest appear in one run and not the other.
The gap between runs varies, because the 42 runs re-collected after the defect described below came back about twenty hours after the originals. Of the 126 run pairs behind that 0.555, 74 sit inside a single collection pass and 52 straddle the gap. We checked whether the gap is doing the work and cannot answer it cleanly from this data. Comparing only the 26 queries that contain both kinds of pair, same-pass pairs average 0.622 and pairs across the gap 0.577, which points the way elapsed time would predict, but only 14 of those 26 queries move in that direction. Pooling all queries reverses the sign, because the 16 queries never re-run at all happen to be less stable than average. The honest summary is that this round cannot separate elapsed time from ordinary run-to-run variance, and a design that collects every run in one pass would be needed to do so.
This finding has published precedent too, and the metric is the same one. In April 2026, Julius Schulte, Malte Bleeker and Philipp Kaufmann at the University of St. Gallen published Don't Measure Once (arXiv 2604.07585, 8 April 2026), which ran repeated queries against ChatGPT, Perplexity, Gemini and Google AI Mode. Their campaign-level mean pairwise Jaccard ran 0.34 to 0.42, with an engine-level range of 0.233 to 0.505 (rank-sensitive RBO, 0.088 to 0.254). Same statistic as ours, so the numbers are comparable.
Two things this round contributes. First, Claude is not in their sample, so our 0.555 adds a fifth engine, and it comes out more stable, sitting above the top of their engine-level range. We are deliberately not turning that into a ranking. Their design was few prompts run many times, 8 prompts per campaign with up to 10 runs each, collected 21 to 25 March 2026 and counting only pairs inside a 24-hour window. Ours is the opposite shape, many queries (50) run few times (3), on one day, on a different query set. "The most stable engine" is not a claim this supports.
Second, and this is the part we think matters more: they did not break the variance down by query type, and we did. Comparison queries came in at 0.435, against 0.620 for recommendations and 0.587 for alternatives. The instability is not spread evenly across the questions your buyers ask. It is concentrated in the ones where two named products are being weighed against each other, which are the same queries where a named brand's own site was retrieved 42.3% of the time.
Disagreement between engines is a different axis, and it has now been measured at a scale hand-run work cannot reach. In July 2026 Writesonic published Do AI Engines Cite the Same Sources?, covering 161,286 prompts across ChatGPT, Gemini, Perplexity and Google AI Overviews, of which 70,879 returned citations from all four at once. Matching cited domains with www. stripped, they report 3.8% of cited domains cited by all four engines and 72 to 73% appearing on exactly one, with per-pair Jaccard running from 0.119 (ChatGPT and Gemini) to 0.237 (Perplexity and Google AI Overviews), and roughly 17% mean pairwise overlap. That is the axis our own earlier study worked on.
Their Jaccard is computed the way ours is, on domain sets, intersection over union. That makes the method the same and the substance different, and the two numbers must not be read against each other. Writesonic measured how far two different engines diverge on one prompt. We measured how far one engine diverges from itself across repeated runs of the same prompt. Their roughly 17% and our 0.555 answer different questions, and setting them side by side would manufacture a "Claude is three times more consistent" claim that neither study supports. Two provenance notes for anyone building on their figures: they publish no raw data, and the collection window, May to June 2026, appears in the page's meta description rather than in the body.
The extremes set the range of what a single measurement can mean. The least stable query in the set was Airtable vs Google Sheets for teams at 0.203, then HubSpot vs Salesforce at 0.239 and Stripe vs PayPal for business at 0.262. All three name two products against each other. At the stable end, best social media management tools held at 0.917, best password manager for teams at 0.905 and best email marketing platform at 0.889.
If you are checking your position on a comparison query by asking once, you are reading one draw from a list that moves. Comparison queries overlapped 0.435 on average, so a meaningful part of the cited list is different the next time you ask. Any claim about "tracking AI visibility" should be held against 0.555 overall, 0.435 on comparisons, and 0.203 on the least stable query we saw.
458 domains retrieved, 333 cited, and a very long tail
Across the study, 458 unique domains were retrieved and 333 of them were cited at least once. Most of what the search returns does get used somewhere.
But look at how thinly it is spread. Of those 333 cited domains, 107 of them, 32%, appeared in exactly one run. They showed up once and never again.
The other end of the distribution, ranked by the number of runs a domain was cited in, then by how many distinct queries it appeared for, then alphabetically. cloudwards.net and mailercloud.com tie on both counts for the last place and the alphabetical rule takes cloudwards.net:
| Domain | Runs cited in | Distinct queries | Times retrieved |
|---|---|---|---|
| zapier.com | 21 | 10 | 34 |
| forbes.com | 17 | 10 | 24 |
| emailvendorselection.com | 13 | 6 | 22 |
| monday.com | 12 | 5 | 18 |
| plutio.com | 12 | 5 | 14 |
| tech-insider.org | 12 | 5 | 14 |
| techradar.com | 11 | 4 | 23 |
| techsifted.com | 10 | 6 | 14 |
| toolradar.com | 10 | 6 | 11 |
| cloudwards.net | 10 | 4 | 11 |
zapier.com leads it: cited in 21 runs across 10 of the 50 queries, retrieved 34 times. forbes.com is second at 17 runs. But the list is not sorted by masthead. emailvendorselection.com was cited in 13 runs, tech-insider.org and plutio.com in 12 each. None of them are household names, and all of them are narrow and specific about one category.
Read the last two columns together as well. tech-insider.org and plutio.com were each retrieved 14 times and cited in 12 runs. emailvendorselection.com was retrieved 22 times and cited in 13. techradar.com was retrieved 23 times and cited in 11. Retrieval and citation do not move together, and the gap between those two columns is the part of the problem that lives on the page itself.
The 32% is the figure to carry around. Close to one cited domain in three appeared exactly once in the whole study. A screenshot of a single citation is a sample, not a standing.
How this sits next to the published work
We would rather line this up ourselves than have a reader do it for us.
| Question | Already published | What this round adds |
|---|---|---|
| Does the engine search at all, by intent? | cloro, Jan 2026: ChatGPT, 100 prompts / 634 answers: 86.5% on commercial, 0.9% on informational, 0.0% across 274 answers in six informational categories and across the 42 answers in "how things work" | The same pattern on Claude, on B2B-SaaS buyer questions, split four ways: 100% / 100% / 100%, and 4.2% on how-to |
| How much of what is retrieved gets cited? | AirOps, Mar 2026: 43,233 queries, 548,534 retrieved pages on ChatGPT, 85% left uncited. And AirOps with Kevin Indig, Apr 2026: 16,851 queries: 58% of pages never cited, 58% citation rate at retrieval position 1 against 14% at position 10 | Claude, unit = domain: 62.7% of retrieved domains cited, median 12 retrieved and 7 cited per run |
| How stable is the cited list across runs? | Schulte et al., St. Gallen, Apr 2026: Jaccard 0.34–0.42 across ChatGPT, Perplexity, Gemini and Google AI Mode | Claude, same statistic: 0.555. And broken out by query type, which they did not do: comparison 0.435, recommendation 0.620, alternatives 0.587 |
| Do different engines cite the same sources for one prompt? | Writesonic, Jul 2026: 161,286 prompts across ChatGPT, Gemini, Perplexity and Google AI Overviews, matched at domain level: 3.8% of cited domains cited by all four, 72 to 73% on exactly one engine, per-pair Jaccard 0.119 to 0.237, roughly 17% mean pairwise overlap | A different axis, not an addition. They measure disagreement between engines on one prompt; this round measures variance within one engine across repeated runs of the same prompt. Their roughly 17% and our 0.555 are different quantities and do not belong on the same scale |
The brand-retrieval result at the top of this page is deliberately not in that table. We have not run a literature survey on that specific question, and not having looked is not the same as there being nothing to find.
What actually follows from this
Five things, each tied to a number above and to nothing else.
Assume the search may not return you even when the buyer names you. In 108 run-brand pairs where the query itself named the brand, the brand's own site was retrieved 36 times, 33.3%, and only 10% on alternatives phrasings. Twelve of 25 brands never appeared once. Your own page is not the asset doing the work on those questions; the pages that do get retrieved are, which makes this an off-page problem before it is an on-page one.
Weight retrieval over selection. Median twelve domains retrieved, median seven cited, 62.7% of retrieved domains cited overall. The second gate discards a minority of what came back; the first one decides what came back at all. Working out which of the two is failing you is the first step when you measure where you stand.
Do not chase citations on "how to" phrasings. One search in 24 runs, and seven of eight how-to queries never searched at all. There is no source list to be in. Spend that effort on the phrasings that searched in 57 of 57, 39 of 39 and 30 of 30 runs.
Never treat one measurement as your position, and least of all on a comparison query. Mean run-to-run overlap was 0.555, comparison queries 0.435, and the least stable query in the set came in at 0.203. A single check tells you what happened once. Our earlier study measured a different disagreement: four engines answering the same question from almost entirely different sources. This one is a single engine, asked three times, on one day.
Treat a first citation as a sample, not a status. 107 of the 333 cited domains, 32% of them, appeared in exactly one run. Repeat appearances across runs and across queries are what is worth measuring; one appearance sits inside the noise. If Claude specifically is the engine you care about, the mechanics are here.
How we ran it
Fifty B2B-SaaS buyer questions across four intents: recommendation, comparison, how-to and alternatives. Each query was run three times, in separate requests, for 150 requests in total.
- Model:
claude-sonnet-5. - Tool: the
web_search_20260209server tool, called directly, withmax_uses: 4. - Tool choice: not set. The search tool was offered on every request and forced on none.
- Location: fixed to the US.
- Collected: 7 September 2026.
- Cost: $0.92 in total. $0.77 of that is read from the Anthropic Console cost report on 7 September 2026: $0.05 on 6 September for the pilot and $0.72 on 7 September for the main run and the first re-run described below. The second re-run added $0.15, read from the credit balance falling from $7.24 to $7.09. Web search was billed at $0.00. An earlier version of this page said $6.05, a figure our own runner had estimated at list prices without the Batch API discount rather than read from billing.
Whether to search was the model's decision on all 150 requests. There is no tool_choice parameter anywhere in the runner. The tool was made available and never compelled, which is the condition that makes the intent table above mean anything. Forcing a tool call produces a search on every run by construction, and a study that forces one is measuring its own configuration: the citations it collects are ones no ordinary user would have triggered. Our own numbers show the tool was not being forced. Recommendation, comparison and alternatives searched in 57 of 57, 39 of 39 and 30 of 30 runs, while how-to searched in 1 of 24. A forced call cannot produce that split. The request shape is in code/run.py in the repository if you want to check it.
For each request we split the response into two lists. Retrieved is every domain that appeared in the search result blocks, meaning what the search handed to the model. Cited is every domain in the answer's citations, meaning what the model chose. Keeping those separate is the whole method; a single overlap percentage collapses the two steps and you lose the ability to tell them apart.
For the brand figures, a run-brand pair is one brand named in one query on one run. The 23 brand-naming queries produced 108 such pairs, and each was scored on one question: was that brand's own site among the retrieved domains?
Runs in which Claude did not search were not discarded. They are recorded as runs with nothing retrieved and nothing cited, which is why the how-to figure exists at all.
The defects we found, and what we did about them
The first version of these results reported zero failed runs. That was wrong, twice over, and both corrections are published here rather than quietly folded into a re-run.
The first defect: 20 runs whose searches returned nothing. In 20 of the 150 runs, a search request was sent but no results came back. The harvesting script was silently skipping the error object, so those runs looked like runs where Claude had simply chosen not to search. The billing trace is what exposed it: those 20 runs had sent 41 searches, of which 0 were billed. The API does not charge for a failed search. The likely cause is that 140 requests went out in a single batch and hit the per-organization search rate limit.
We fixed the script (it now records a search_errors field), re-ran all 20, and every one returned results. The faulty first version of the findings file has been kept, not deleted.
The second defect: 42 runs with incomplete retrieval. The rule that caught the first one, that searches issued should equal searches billed, was written into our own defect record and then applied only to the 20 runs already under suspicion. It was never run across the other 130. When we did apply it to all 150, 42 runs failed it: they had issued more searches than were billed, which means some of their searches returned nothing at all. Those runs did have results, so nothing looked wrong in the output, but they were short of part of what they had asked for and they went into the analysis with incomplete retrieval.
We re-ran all 42, in three batches of 14 rather than in one batch, since one large batch is what broke the searches the first time. All 150 runs now pass the check: searches issued equals searches billed, no error objects, no empty result sets.
A re-run is a fresh search, not a repair of the old one. The 42 replacements were collected about twenty hours after the originals, so the dataset covers two collection times, both on 7 September 2026 local time. Figures moved for two reasons at once and the two cannot be separated: retrieval that was missing is now present, and the web returned different results later in the day.
What moved. The headline brand-retrieval rate went up, from 28.7% to 33.3%, 31 of 108 run-brand pairs to 36 of 108, and comparison from 35.9% to 42.3%. The named brand's own site came back more often after the re-run, not less. The share of retrieved domains that got cited fell from 68.3% to 62.7%, the largest single move on this page: the defective runs were short of retrieved domains, and retrieved domains are the denominator, so that rate had been inflated. Run-to-run overlap also moved up, from 0.506 to 0.555 overall and from 0.370 to 0.435 on comparison queries, so Claude comes out more stable than the first version of this page said, not less. The least stable query in the set is now 0.203 rather than 0.030, and Slack vs Microsoft Teams, which the first version used as its worst case, is now 0.355. The never-retrieved brand list stayed at 12 of 25 but changed membership, as described further up this page. Unique domains retrieved went from 441 to 458, cited from 332 to 333, total citations from 2,090 to 2,138, and cited domains appearing in exactly one run from 125 (38%) to 107 (32%).
What did not move. The intent table is unchanged: searches fired in 127 of 150 runs, 57 of 57 recommendation, 39 of 39 comparison, 30 of 30 alternatives and 1 of 24 how-to. None of the 42 defective runs was a how-to run. The alternatives brand-retrieval rate is still 3 of 30.
Every figure on this page is computed from the 150 runs as they stand after both re-runs.
The raw per-request JSON is published, one file per request, with the full response, the
citations and the usage, in the study's repository. Alongside it: the 150 runs as one row
each in data/runs.csv, every retrieved URL flagged with whether it was cited in data/urls.csv,
the brand-retrieval rows behind the headline figure in data/brand-retrieval.csv, the runner in
code/run.py, and the protocol and defect record in notes/. Everything is CC BY 4.0.
What this round did not measure
Seven limits, stated plainly.
1. There is no Brave comparison, so the index-inheritance question was not tested. The Brave top-10 for these 50 queries was not collected. Whatever you may have read about AI engines inheriting the rankings of an underlying search index, this round does not answer that question and does not claim to.
2. The surface we measured is the Claude API's web search tool, not claude.ai. Whether the two use the same search provider is not documented by Anthropic. These findings describe the API's web search tool. We are not generalizing them to the consumer product, and doing so would need a separate round.
3. One model, one language, one vertical. claude-sonnet-5, English, B2B-SaaS buyer questions.
4. Location was fixed to the US, and search was capped at max_uses: 4. Results may differ from another location. The highest search count we observed in any run was 3, so the cap did not bind in practice, but it was in place.
5. Runs whose searches came back incomplete were re-run, twice. Twenty runs' searches failed silently in the first pass and were re-run about twenty minutes later. A later check across all 150 runs found 42 in which more searches had been issued than were billed, meaning part of the retrieval was missing; those were re-run in three batches of 14 the same evening, and all 150 runs now pass. A replacement run is a fresh search rather than a repair of the old one, so the index may have moved between a run and the run that replaced it. Given that the same query run twice overlaps at 0.555 anyway, we would not read anything into a single domain's presence or absence; the aggregate figures are what this study supports.
6. An earlier version of this analysis led with a finding that we removed before publishing. That version's headline was that Claude never cited a domain its own search had not returned, with no exceptions across the 127 searching runs. An adversarial review before publication took it apart. All 2,138 citations matched a retrieved URL exactly, and each web search citation carries a reference back into the search results, so a citation cannot point at something the search did not return. The count was real; the interpretation was not. It measured the API's citation format rather than anything Claude does, which makes it a tautology and not a finding, so it is gone. We are recording that we caught it for the same reason we published the 20 failed runs above.
7. Three runs per query is fewer than the published guidance for a stable point estimate. The St. Gallen paper cited above ran a bootstrap convergence analysis on its 10-run dataset and reports that the standard error of a per-brand detection rate drops below 0.10 at 7 runs and below 0.08 at 8, while source-level coverage converges more slowly and needs 8 runs to reach a standard error under 0.10. Their recommendation is at least 7 runs per prompt per day for brand monitoring, and at least 8 when source-level coverage is the thing being measured. We ran 3. So treat 33.3%, 62.7%, 4.2% and the per-intent Jaccard means as point estimates carrying wider intervals than an 8-run design would give, and do not read a small gap between two of our cells as a real difference.
The run-to-run result is the one figure this does not soften. It measures spread rather than position, and three runs were already enough for the spread to show itself: 0.555 overall, 0.435 on comparison queries, 0.203 on the least stable query in the set. More runs would describe that distribution more finely. They would not make the cited list move less.
Cite this study
This is original, hand-run research and you are welcome to cite or share it (CC BY 4.0). Suggested attribution:
Is My Brand in AI (2026). The Claude Retrieval and Citation Study, round one: retrieval and selection in the Claude API's web search tool. https://ismybrandinai.com/claude-retrieval-and-citation-study
Fifty B2B-SaaS buyer questions, three runs each, 150 requests against claude-sonnet-5 with the web_search_20260209 tool called directly, collected 7 September 2026.
The raw runs are published, not available on request. The same files are archived at CERN's Zenodo under doi.org/10.5281/zenodo.22651233, which is citable and cannot be quietly edited. If a count here does not match what you get from the files, tell us and we will publish a correction saying what changed.
Work this study builds on
- cloro (Ricardo Batista), ChatGPT Grounding Study, 14 January 2026. https://cloro.dev/blog/chatgpt-grounding-frequency-study/
- AirOps with Kevin Indig, The Fan-Out Effect: What Happens Between a Query and a Citation, 13 April 2026. https://www.airops.com/report/the-fan-out-effect-what-happens-between-a-query-and-a-citation
- AirOps, The Influence of Retrieval, Fan-out, and Google SERPs on ChatGPT Citations, 12 March 2026. https://www.airops.com/report/influence-of-retrieval-fanout-and-google-serps-in-chatgpt
- Schulte, Bleeker and Kaufmann (University of St. Gallen), Don't Measure Once, arXiv 2604.07585, 8 April 2026. https://arxiv.org/abs/2604.07585
- Writesonic (Samanyou Garg), Do AI Engines Cite the Same Sources? We Studied 161,286 Prompts Across 4 Platforms, 22 July 2026. https://writesonic.com/blog/ai-citation-source-overlap-study
- I. Strauss, J. Yang, T. O'Reilly, S. Rosenblat and I. Moure, The Attribution Crisis in LLM Search Results, arXiv 2508.00838, 27 June 2025. https://arxiv.org/abs/2508.00838
- S. Kale, Look It Up: Analysing Internal Web Search Capabilities of Modern LLMs, arXiv 2511.18931, 24 November 2025. https://arxiv.org/abs/2511.18931