Open data
The AI Answer Evidence Index: the tables
Every table behind the two articles, for the five engines. This page is a rendering of tables.md, which report_tables.py writes from the outputs of the chain. All files.
Tables
- 1. The frame: 50 questions per engine
- 2. Primary measure: share of an answer's figures found on a page the answer cites
- 3. Outcomes of every figure
- 4. Pooled shares of figures, and the views beside them
- 5. The classifier's labels for the figures not found
- 6. Found on any cited page, by sector
- 7. Validation sample: 40 figures read by Claude Opus 5.5 (claude-opus-5-5) on 2026-10-07
- 8. Repeat run (Google AI Overviews, ChatGPT and Gemini): figures found in the first and in the second answer to the same question
- S1. Median share per answer: the same questions for all engines, and other sets
- S1b. Paired differences on the same questions
- S2. Pages per answer, and figures found by the number of pages
- S2b. What one page contributes: the pair share and the cuts
- S3. Placebo: pages cited for questions of other sectors
- S4. Units, attached sources, and figures not on a readable page
- S5. Sector cells: answers, and figures of answers that cite no source
- S6. The like-for-like comparison in four views
- A1. Sources per engine
- A2. Overlap of the sources cited for the same question
- A2b. Sources by the number of engines that cite them
- A3. What the list sizes allow: the ceiling of the all-engine overlap
- A4. Overlap by pair of engines: Jaccard and containment
- A5. Yardstick (Google AI Overviews, ChatGPT and Gemini): the same engine asked the same question twice (pages)
- A5b. The yardstick like for like (pages)
- A6. Most cited domains, five engines
Written by report/report_tables.py from the outputs of the frozen scripts. Five engines in one order: Google AI Overviews, ChatGPT, Gemini, Perplexity, Claude (API). The preregistration of 5 October 2026 names Google AI Overviews, ChatGPT and Gemini. Perplexity and Claude (API) were added on 6 October, and their answers went through the same four steps on 7 and 8 October, after the results of the first three had been computed, as an exploratory addition (deviation log, entries 3, 5, 6 and 7); see the method. The values of the three engines the preregistration names are printed unchanged. No percentage is given for a cell with fewer than 5 units, and no median for fewer than 5 answers. A count that is not zero and would print as 0% is given without a percentage.
1. The frame: 50 questions per engine
| Engine | Answered | No AI Overview | No answer | Parse failed | No record | Answers with figures | Answers without figures | Extraction failed | Figures | Left unlabelled | First answer (UTC+3) | Last answer (UTC+3) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Google AI Overviews | 49 | 0 | 1 | 0 | 0 | 47 | 2 | 0 | 640 | 0 | 5 October 2026, 16:34 | 7 October 2026, 00:08 |
| ChatGPT | 50 | 0 | 0 | 0 | 0 | 47 | 3 | 0 | 470 | 0 | 5 October 2026, 16:53 | 6 October 2026, 23:21 |
| Gemini | 50 | 0 | 0 | 0 | 0 | 50 | 0 | 0 | 661 | 0 | 5 October 2026, 17:08 | 6 October 2026, 22:34 |
| Perplexity | 50 | 0 | 0 | 0 | 0 | 47 | 3 | 0 | 755 | 0 | 6 October 2026, 17:07 | 6 October 2026, 22:08 |
| Claude (API) | 50 | 0 | 0 | 0 | 0 | 50 | 0 | 0 | 939 | 0 | 6 October 2026, 22:39 | 6 October 2026, 22:44 |
All five engines: 249 of 250 answers collected, 241 answers with figures, 3465 figures, 2693 of 3465 (78%) of them found. Google AI Overviews, ChatGPT and Gemini: 149 of 150 answers collected, 144 answers with figures, 1771 figures, 1227 of them found. Perplexity and Claude (API): 100 of 100 answers collected, 97 answers with figures, 1694 figures, 1466 of them found.
ChatGPT answers that cite no source: 6 (BIZ-04, SVC-06, SVC-07, SVC-09, SVC-10, TRV-08). Their saved pages hold 0 "Sources" controls and 0 outbound links (a sourced answer, AIT-03: 2 and 59), which indicates that ChatGPT answered them without a search. 2 of them state no figure (BIZ-04, SVC-07); the other 4 (SVC-06, SVC-09, SVC-10, TRV-08) hold 28 figures. Claude (API) answers that ran no search and cite no source: 2 (TRV-08, TRV-16), with 13 figures. Perplexity: every answer cites a source.
Cited pages. The answers of Google AI Overviews, ChatGPT and Gemini cite 483 addresses, 410 readable as the check counts them (a fetch returned the page's text, and the text is not a stub or a check page). A fetch returned text for 429 of 483 (89%) addresses, the count of the run record; the check sets aside 19 of them, because a fetched text under 500 characters, or a check page, is not the page's text (19 with every text under 500 characters, 0 with a check page). Each of these addresses was fetched 0.1 to 2.0 hours after the first answer that cites it.
The answers of Perplexity and Claude (API) cite 722 addresses, 601 of 722 (83%) readable as the check counts them. 147 of 722 addresses reuse the fetch made for the main and the repeat set (144 readable), made from 5 October 2026, 17:28 to 7 October 2026, 14:10 (UTC+3): a reused fetch lies between 29 hours before and 21 hours after the answer of Perplexity or Claude (API) that cites it, and 108 of 147 reused addresses were fetched before such an answer (107 of 147 before every such answer). The other 575 of 722 were fetched from 7 October 2026, 23:57 to 8 October 2026, 04:56 (UTC+3) (457 readable): 25 to 34 hours after the answers that cite them. The four steps on these answers ran from 7 October 2026, 23:41 to 8 October 2026, 05:07 (addendum/figures/chain.log, the machine's time, UTC+3).
All five engines: 1045 distinct addresses; 160 of them are cited both by one of the first three engines and by Perplexity or Claude (API). 20 of those had not been readable in the fetch of the main set and were fetched again; 7 of the 20 were readable in the later fetch. Each answer is checked against the fetch of its own set.
Perplexity: 608 source entries (every entry of the source list the interface shows; a median of 10 per answer), 553 distinct addresses. The records hold host and path without the query string: 22 of 608 source entries had a query string that was not kept (19 addresses in 14 answers), and their pages are fetched without it, so the page read for them may differ from the page cited; 70 of 644 (11%) of the found figures were found on such a page and 2 of 644 only on such pages. No record ties a passage to a source address, so no figure has an attached source. 534 citation chip lines (394 site-name lines and 140 counter lines) are left out of the answer text before the extraction. 5 of 50 records carry the interface's notice that a preview of the advanced search was switched on; the notes of 10 further records say that the notice could not be observed. Claude (API): 272 source entries (the pages that a citation of the response names), 236 distinct addresses; 48 of 50 answers have citations (524 cited text blocks, 537 citation entries).
Time stamps: every one carries the offset +0300 (UTC+3). Main set, first and last answer: 2026-10-05T16:34:54+0300 and 2026-10-07T00:08:15+0300. Repeat set: 2026-10-07T00:25:54+0300 and 2026-10-07T01:23:51+0300.
2. Primary measure: share of an answer's figures found on a page the answer cites
Each engine on its own answers with figures.
| Engine | Answers with figures | Median share per answer | 95% bootstrap interval |
|---|---|---|---|
| Google AI Overviews | 47 | 82% | 72% to 92% |
| ChatGPT | 47 | 55% | 40% to 69% |
| Gemini | 50 | 73% | 66% to 83% |
| Perplexity | 47 | 94% | 86% to 100% |
| Claude (API) | 50 | 93% | 89% to 98% |
The rows of Google AI Overviews, ChatGPT and Gemini are the primary measure as registered. Perplexity and Claude (API) were added after the preregistration; see the method. Each engine stands on its own questions here; the comparison on the same questions is in table S1.
Gemini without the eight answers of block 1 that were saved after the block's last passing exit check (deviation 1): 42 answers, median 69%, interval 61% to 85%.
Perplexity without its answer to SVC-07, which was read from the account's stored thread after an interruption (deviation log, run record of 6 October 2026): 46 answers, median 93%, interval 86% to 100%, 641 of 752 (85%) figures found; with it, 47 answers, 94% (86% to 100%), 644 of 755 (85%). The answer holds 3 figures, 3 of them found. SVC-07 is not among the questions of the like-for-like row of table S1 (no figure in the answer of ChatGPT), so that row is the same with and without it.
3. Outcomes of every figure
| Engine | Figures | Found on the attached page | Found on another cited page | Not found | No source cited | Page not readable | Instrument gap | Unknown: no source cited, page not readable or instrument gap | Found on any cited page |
|---|---|---|---|---|---|---|---|---|---|
| Google AI Overviews | 640 | 202 of 640 (32%) | 305 of 640 (48%) | 34 of 640 (5%) | 15 of 640 (2%) | 83 of 640 (13%) | 1 of 640 | 99 of 640 (15%) | 507 of 640 (79%) |
| ChatGPT | 470 | 127 of 470 (27%) | 140 of 470 (30%) | 109 of 470 (23%) | 28 of 470 (6%) | 66 of 470 (14%) | 0 of 470 (0%) | 94 of 470 (20%) | 267 of 470 (57%) |
| Gemini | 661 | 281 of 661 (43%) | 172 of 661 (26%) | 137 of 661 (21%) | 10 of 661 (2%) | 56 of 661 (8%) | 5 of 661 (1%) | 71 of 661 (11%) | 453 of 661 (69%) |
| Perplexity | 755 | not available | 644 of 755 (85%) | 2 of 755 | 0 of 755 (0%) | 107 of 755 (14%) | 2 of 755 | 109 of 755 (14%) | 644 of 755 (85%) |
| Claude (API) | 939 | 574 of 939 (61%) | 248 of 939 (26%) | 32 of 939 (3%) | 13 of 939 (1%) | 71 of 939 (8%) | 1 of 939 | 85 of 939 (9%) | 822 of 939 (88%) |
| All five engines | 3465 | 1184 | 1509 | 314 | 66 | 383 | 9 | 458 | 2693 |
| Google AI Overviews, ChatGPT and Gemini | 1771 | 610 | 617 | 280 | 53 | 205 | 6 | 264 | 1227 |
Perplexity: its records tie no passage to a source address, so the attached view is not available and every found figure stands under "found on another cited page", which there means found on any cited page.
4. Pooled shares of figures, and the views beside them
| View | Google AI Overviews | ChatGPT | Gemini | Perplexity | Claude (API) |
|---|---|---|---|---|---|
| Found on any cited page | 507 of 640 (79%) | 267 of 470 (57%) | 453 of 661 (69%) | 644 of 755 (85%) | 822 of 939 (88%) |
| Found on the attached page | 202 of 640 (32%) | 127 of 470 (27%) | 281 of 661 (43%) | not available | 574 of 939 (61%) |
| Found, without the unknowns | 507 of 556 (91%) | 267 of 404 (66%) | 453 of 600 (76%) | 644 of 646 (100%) | 822 of 867 (95%) |
| Found on the attached page, without the unknowns | 202 of 556 (36%) | 127 of 404 (31%) | 281 of 600 (47%) | not available | 574 of 867 (66%) |
| Found, in answers whose cited pages were all readable | 268 of 302 (89%) | 217 of 326 (67%) | 347 of 489 (71%) | 61 of 63 (97%) | 397 of 430 (92%) |
| Found, without figures whose marker names an unseen page | 507 of 640 (79%) | 197 of 365 (54%) | 411 of 613 (67%) | not recorded | not recorded |
| Found, without answers that hold a Sponsored label | 507 of 640 (79%) | 267 of 470 (57%) | 453 of 661 (69%) | not recorded | not recorded |
The collections of Perplexity and Claude (API) recorded neither markers that name an unlisted page nor Sponsored labels. Answers whose saved page holds a "Sponsored" label: Google AI Overviews 2 on the page, 0 inside the answer text; ChatGPT 0 on the page, 0 inside the answer text; Gemini 0 on the page, 0 inside the answer text.
5. The classifier's labels for the figures not found
| Engine | Not found | Absent | Partial | Rounded | Derived | Present | Unlabelled | Absent only because the quote was not confirmed |
|---|---|---|---|---|---|---|---|---|
| Google AI Overviews | 34 | 18 | 15 | 1 | 0 | 0 | 0 | 1 |
| ChatGPT | 109 | 86 | 21 | 2 | 0 | 0 | 0 | 4 |
| Gemini | 137 | 80 | 51 | 5 | 0 | 1 | 0 | 5 |
| Perplexity | 2 | 0 | 2 | 0 | 0 | 0 | 0 | 0 |
| Claude (API) | 32 | 19 | 8 | 5 | 0 | 0 | 0 | 1 |
| All five engines | 314 | 203 | 97 | 13 | 0 | 1 | 0 | 11 |
| Google AI Overviews, ChatGPT and Gemini | 280 | 184 | 87 | 8 | 0 | 1 | 0 | 10 |
All five engines: 203 of 314 figures that were not found are labelled absent, 11 of them only because the quote was not confirmed.
6. Found on any cited page, by sector
| Sector | Google AI Overviews | ChatGPT | Gemini | Perplexity | Claude (API) |
|---|---|---|---|---|---|
| AI tools | 112 of 113 (99%) | 78 of 85 (92%) | 87 of 88 (99%) | 115 of 116 (99%) | 118 of 119 (99%) |
| Business software | 71 of 90 (79%) | 40 of 92 (43%) | 118 of 157 (75%) | 200 of 250 (80%) | 229 of 240 (95%) |
| Consumer electronics | 61 of 89 (69%) | 51 of 89 (57%) | 32 of 74 (43%) | 107 of 117 (91%) | 104 of 127 (82%) |
| Legal and local services | 111 of 135 (82%) | 25 of 65 (38%) | 106 of 150 (71%) | 90 of 105 (86%) | 168 of 182 (92%) |
| Personal finance | 89 of 122 (73%) | 55 of 84 (65%) | 65 of 120 (54%) | 80 of 98 (82%) | 126 of 162 (78%) |
| Travel | 63 of 91 (69%) | 18 of 55 (33%) | 45 of 72 (62%) | 52 of 69 (75%) | 77 of 109 (71%) |
A sector cell holds 7 to 9 answers with figures (table S5). The cells of Perplexity and Claude (API) were also counted from their check outputs and equal the cells of their summary.
7. Validation sample: 40 figures read by Claude Opus 5.5 (claude-opus-5-5) on 2026-10-07
The sample was drawn from the figures of Google AI Overviews, ChatGPT and Gemini, the three engines the preregistration names. Figures of Perplexity or Claude (API) in the sample: 0.
| Stratum | Reading | Figures |
|---|---|---|
| Found (20) | supports | 19 |
| Found (20) | coincidental | 1 |
| Not found (20) | partial | 6 |
| Not found (20) | absent | 11 |
| Not found (20) | derived | 2 |
| Not found (20) | present | 1 |
Classifier and reader gave the same label to 16 of 20 not-found figures, and agreed on absent or not absent for 16 of 20. Unlabelled: 0.
| Reader | Classifier | Figures |
|---|---|---|
| absent | absent | 11 |
| derived | absent | 2 |
| partial | absent | 2 |
| partial | partial | 4 |
| present | present | 1 |
| Stratum | Google AI Overviews | ChatGPT | Gemini |
|---|---|---|---|
| Found (20) | 6 | 3 | 11 |
| Not found (20) | 1 | 9 | 10 |
Readings with a quote that the script confirmed in the saved page: 35 of 40 (88%). Readings without a quote: 5, all read as absent.
Sampled figures the classifier called absent: 15; the reader read 4 of 15 of them as something else (derived 2, partial 2). These are all 4 disagreements of the not-found sample.
Found figures of the sample that carry no unit: 4 of 20 (20%), read as supports (3), coincidental (1). The label "supports weakly" was given to 0 figures.
8. Repeat run (Google AI Overviews, ChatGPT and Gemini): figures found in the first and in the second answer to the same question
The first ten questions of the asking order, asked a second time of the three engines the preregistration names. Perplexity and Claude (API) were asked each question once. Not pooled with the main result. "No figures" also stands for a question without a complete answer.
| Question | Google AI Overviews, first | Google AI Overviews, second | ChatGPT, first | ChatGPT, second | Gemini, first | Gemini, second |
|---|---|---|---|---|---|---|
| BIZ-04 | 9 of 11 (82%) | 8 of 8 (100%) | no figures | 23 of 24 (96%) | 13 of 20 (65%) | 8 of 17 (47%) |
| TRV-05 | no figures | no figures | 0 of 2 | 1 of 1 | 6 of 8 (75%) | 4 of 5 (80%) |
| AIT-04 | 25 of 25 (100%) | 17 of 17 (100%) | 15 of 15 (100%) | 16 of 16 (100%) | 18 of 18 (100%) | 20 of 20 (100%) |
| FIN-01 | 12 of 18 (67%) | 10 of 12 (83%) | 0 of 7 (0%) | 0 of 4 | 12 of 12 (100%) | 16 of 16 (100%) |
| FIN-02 | 11 of 12 (92%) | 10 of 19 (53%) | 2 of 4 | 3 of 6 (50%) | 11 of 16 (69%) | 15 of 20 (75%) |
| ELC-07 | no figures | no figures | 2 of 2 | 3 of 5 (60%) | 3 of 4 | 1 of 1 |
| AIT-17 | 3 of 3 | 3 of 3 | no figures | 0 of 1 | 3 of 3 | 3 of 3 |
| TRV-06 | 15 of 21 (71%) | 16 of 18 (89%) | 2 of 8 (25%) | 4 of 10 (40%) | 9 of 12 (75%) | 11 of 15 (73%) |
| SVC-08 | 9 of 13 (69%) | 5 of 10 (50%) | 5 of 6 (83%) | 0 of 6 (0%) | 20 of 22 (91%) | 26 of 30 (87%) |
| ELC-16 | 0 of 15 (0%) | 0 of 13 (0%) | 4 of 12 (33%) | 4 of 12 (33%) | 5 of 17 (29%) | 14 of 22 (64%) |
| Questions with figures in both answers | 84 of 118 (71%), 8 questions | 69 of 100 (69%) | 30 of 56 (54%), 8 questions | 31 of 60 (52%) | 100 of 132 (76%), 10 questions | 118 of 149 (79%) |
Repeat answers by engine: Google AI Overviews 8 answers, 100 figures, 69 of 100 (69%) found; ChatGPT 10 answers, 85 figures, 54 of 85 (64%) found; Gemini 10 answers, 149 figures, 118 of 149 (79%) found. Complete second answers: 28 of 30.
Views beside the primary measure
Written from report/side-views.json (report/side_views.py). Additional computations: they stand beside the tables above and replace none of them. Every interval follows the registered rule (10,000 resamples of questions, seed 1109872877, percentile bounds).
S1. Median share per answer: the same questions for all engines, and other sets
Like for like: the same questions for every engine of the row. Each engine on its own answers: the medians of a row are not compared with each other. Perplexity and Claude (API) were added after the preregistration; see the method.
| Set | Answers | Google AI Overviews | ChatGPT | Gemini | Perplexity | Claude (API) |
|---|---|---|---|---|---|---|
| Like for like: the 41 questions where all five answers state a figure | 41, 41, 41, 41, 41 | 82% (72% to 92%) | 57% (42% to 75%) | 71% (67% to 91%) | 92% (86% to 100%) | 91% (88% to 96%) |
| Like for like: figures that carry a unit only, on the 39 questions where all five answers hold one | 39, 39, 39, 39, 39 | 80% (67% to 91%) | 50% (27% to 67%) | 70% (47% to 90%) | 90% (78% to 100%) | 92% (85% to 100%) |
| Like for like: the 35 questions where all five answers state a figure and cite a page | 35, 35, 35, 35, 35 | 82% (71% to 92%) | 57% (43% to 82%) | 78% (68% to 92%) | 88% (75% to 95%) | 91% (88% to 97%) |
| Like for like, three engines: the 44 questions where the answers of Google AI Overviews, ChatGPT and Gemini state a figure | 44, 44, 44 | 83% (73% to 92%) | 55% (40% to 69%) | 71% (65% to 85%) | not in the set | not in the set |
| Like for like, three engines: the 38 questions where those three answers state a figure and cite a page | 38, 38, 38 | 83% (72% to 93%) | 57% (43% to 76%) | 73% (67% to 91%) | not in the set | not in the set |
| Each engine on its own answers: all answers with figures (table 2) | 47, 47, 50, 47, 50 | 82% (72% to 92%) | 55% (40% to 69%) | 73% (66% to 83%) | 94% (86% to 100%) | 93% (89% to 98%) |
| Each engine on its own answers: answers that cite at least one page | 46, 43, 49, 47, 48 | 83% (74% to 92%) | 57% (42% to 75%) | 75% (67% to 87%) | 94% (86% to 100%) | 94% (89% to 100%) |
| Each engine on its own answers: answers with two or more readable pages | 46, 22, 38, 47, 48 | 83% (74% to 92%) | 76% (55% to 90%) | 76% (70% to 96%) | 94% (86% to 100%) | 94% (89% to 100%) |
| Each engine on its own answers: answers whose cited pages were all readable | 18, 30, 36, 5, 25 | 95% (81% to 100%) | 61% (46% to 87%) | 75% (66% to 91%) | 100% (88% to 100%) | 100% (91% to 100%) |
| Each engine on its own answers: figures that carry a unit only | 44, 41, 48, 45, 47 | 80% (67% to 91%) | 50% (27% to 60%) | 69% (47% to 90%) | 92% (83% to 100%) | 94% (88% to 100%) |
All figures. Intervals on the 41 questions. Do not overlap: ChatGPT and Perplexity; ChatGPT and Claude (API). The other 8 of the 10 pairs overlap.
Figures that carry a unit only. Intervals on the 39 questions. Do not overlap: ChatGPT and Perplexity; ChatGPT and Claude (API). Meet at one value: Google AI Overviews and ChatGPT. The other 7 of the 10 pairs overlap. The same 39 questions are the ones where the answers of Google AI Overviews, ChatGPT and Gemini alone hold a figure with a unit.
Answers that state a figure and cite a page. Intervals on the 35 questions. Do not overlap: ChatGPT and Claude (API). The other 9 of the 10 pairs overlap.
Google AI Overviews and ChatGPT: on the 41 questions of the five engines their intervals overlap; on the 44 questions of the three-engine row they do not overlap; on the 38 questions they overlap; for the figures that carry a unit, on the 39 questions, they meet at one value. The 41 questions are the 44 without AIT-12, FIN-09, TRV-08: Perplexity states no figure for AIT-12, FIN-09, TRV-08. Figures found in the answers to those questions (Google AI Overviews, ChatGPT, Gemini): AIT-12: 3 of 3, 1 of 3, 1 of 2; FIN-09: 7 of 11 (64%), 0 of 1, 3 of 7 (43%); TRV-08: 2 of 2, 0 of 2, 6 of 8 (75%).
Answers with figures that cite no page: Google AI Overviews ELC-16; ChatGPT SVC-06, SVC-09, SVC-10, TRV-08; Gemini ELC-13; Perplexity none; Claude (API) TRV-08, TRV-16.
S1b. Paired differences on the same questions
The first engine minus the second on the same questions. Each resample draws one list of questions and uses it for both engines. Differences are in percentage points.
Every pair of the five engines on the 41 questions where all five answers state a figure. Each pair is written with the engine of the higher median first.
| Pair | Questions | Statistic | Median, first | Median, second | Difference | 95% interval of the difference | First higher | Lower | Equal |
|---|---|---|---|---|---|---|---|---|---|
| Google AI Overviews minus ChatGPT | 41 | difference of the medians | 82% | 57% | +25 points | +8 to +42 points | 27 of 41 | 7 of 41 | 7 of 41 |
| Google AI Overviews minus Gemini | 41 | difference of the medians | 82% | 71% | +11 points | -5 to +19 points | 21 of 41 | 12 of 41 | 8 of 41 |
| Perplexity minus Google AI Overviews | 41 | difference of the medians | 92% | 82% | +10 points | +0 to +19 points | 19 of 41 | 13 of 41 | 9 of 41 |
| Claude (API) minus Google AI Overviews | 41 | difference of the medians | 91% | 82% | +9 points | +1 to +19 points | 22 of 41 | 14 of 41 | 5 of 41 |
| Gemini minus ChatGPT | 41 | difference of the medians | 71% | 57% | +14 points | -0 to +36 points | 23 of 41 | 11 of 41 | 7 of 41 |
| Perplexity minus ChatGPT | 41 | difference of the medians | 92% | 57% | +35 points | +18 to +50 points | 30 of 41 | 3 of 41 | 8 of 41 |
| Claude (API) minus ChatGPT | 41 | difference of the medians | 91% | 57% | +34 points | +16 to +51 points | 30 of 41 | 5 of 41 | 6 of 41 |
| Perplexity minus Gemini | 41 | difference of the medians | 92% | 71% | +20 points | +1 to +31 points | 23 of 41 | 10 of 41 | 8 of 41 |
| Claude (API) minus Gemini | 41 | difference of the medians | 91% | 71% | +19 points | +3 to +26 points | 23 of 41 | 11 of 41 | 7 of 41 |
| Perplexity minus Claude (API) | 41 | difference of the medians | 92% | 91% | +1 points | -7 to +9 points | 17 of 41 | 15 of 41 | 9 of 41 |
Google AI Overviews minus ChatGPT on other sets of questions:
| Questions of the set | Questions | Statistic | Median, first | Median, second | Difference | 95% interval of the difference | First higher | Lower | Equal |
|---|---|---|---|---|---|---|---|---|---|
| The 44 questions where the answers of Google AI Overviews, ChatGPT and Gemini state a figure | 44 | difference of the medians | 83% | 55% | +28 points | +14 to +46 points | 30 of 44 | 7 of 44 | 7 of 44 |
| Questions where both answers state a figure and cite a page | 39 | difference of the medians | 82% | 57% | +25 points | +6 to +40 points | 26 of 39 | 6 of 39 | 7 of 39 |
| Figures that carry a unit only, questions where both answers hold one | 39 | difference of the medians | 80% | 50% | +30 points | +14 to +48 points | 24 of 39 | 7 of 39 | 8 of 39 |
| Questions where both answers have two or more readable pages | 20 | mean of the per-question differences | 88% | 76% | +15 points | +6 to +25 points | 12 of 20 | 2 of 20 | 6 of 20 |
S2. Pages per answer, and figures found by the number of pages
Each engine on its own answers with figures. Pooled counts of figures, with the number of answers of the cell.
| View | Google AI Overviews | ChatGPT | Gemini | Perplexity | Claude (API) |
|---|---|---|---|---|---|
| Median cited pages per answer | 7 | 2 | 3 | 10 | 6 |
| Median readable pages per answer | 6 | 1 | 2 | 9 | 5 |
| Answers with figures that have at most one readable page | 1 of 47 (2%) | 25 of 47 (53%) | 12 of 50 (24%) | 0 of 47 (0%) | 2 of 50 (4%) |
| Figures found in those answers | 0 of 15 (0%), 1 answer | 81 of 208 (39%), 25 answers | 43 of 120 (36%), 12 answers | no answer | 0 of 13 (0%), 2 answers |
| Of those answers, with a cited page that could not be read | 0 of 1 | 9 of 25 (36%) | 5 of 12 (42%) | no answer | 0 of 2 |
| Answers with no readable page: found | 0 of 15 (0%), 1 answer | 0 of 37 (0%), 6 answers | 0 of 35 (0%), 4 answers | no answer | 0 of 13 (0%), 2 answers |
| Answers with one readable page: found | no answer | 81 of 171 (47%), 19 answers | 43 of 85 (51%), 8 answers | no answer | no answer |
| Answers with two readable pages: found | 39 of 55 (71%), 5 answers | 117 of 165 (71%), 14 answers | 143 of 193 (74%), 17 answers | 2 of 10 (20%), 1 answer | 44 of 51 (86%), 2 answers |
| Answers with three or more readable pages: found | 468 of 570 (82%), 41 answers | 69 of 97 (71%), 8 answers | 267 of 348 (77%), 21 answers | 642 of 745 (86%), 46 answers | 778 of 875 (89%), 46 answers |
| Answers with two or more readable pages: found | 507 of 625 (81%), 46 answers | 186 of 262 (71%), 22 answers | 410 of 541 (76%), 38 answers | 644 of 755 (85%), 47 answers | 822 of 926 (89%), 48 answers |
| Answers with 3 to 4 readable pages: found | 112 of 131 (85%), 9 answers | 54 of 77 (70%), 7 answers | 232 of 310 (75%), 18 answers | no answer | 221 of 265 (83%), 16 answers |
| Answers with 5 to 7 readable pages: found | 183 of 230 (80%), 18 answers | 15 of 20 (75%), 1 answer | 35 of 38 (92%), 3 answers | 89 of 105 (85%), 9 answers | 557 of 610 (91%), 30 answers |
| Answers with 8 or more readable pages: found | 173 of 209 (83%), 14 answers | no answer | no answer | 553 of 640 (86%), 37 answers | no answer |
| Answers that cite 1 page: found | no answer | 57 of 118 (48%), 14 answers | 37 of 82 (45%), 7 answers | no answer | no answer |
| Answers that cite 2 pages: found | 9 of 11 (82%), 1 answer | 120 of 188 (64%), 18 answers | 105 of 157 (67%), 14 answers | no answer | no answer |
| Answers that cite 3 to 4 pages: found | 123 of 145 (85%), 9 answers | 70 of 110 (64%), 9 answers | 264 of 362 (73%), 24 answers | no answer | 118 of 130 (91%), 9 answers |
| Answers that cite 5 to 7 pages: found | 155 of 192 (81%), 16 answers | 20 of 26 (77%), 2 answers | 47 of 50 (94%), 4 answers | no answer | 682 of 772 (88%), 37 answers |
| Answers that cite 8 or more pages: found | 220 of 277 (79%), 20 answers | no answer | no answer | 644 of 755 (85%), 47 answers | 22 of 24 (92%), 2 answers |
Rank correlation of the number of pages of an answer with its share found (Spearman, with the interval of the registered resampling rule):
| Answers | Google AI Overviews | ChatGPT | Gemini | Perplexity | Claude (API) |
|---|---|---|---|---|---|
| One or more readable pages, by readable pages | +0.03 (-0.25 to +0.32), 46 answers | +0.34 (+0.06 to +0.58), 41 answers | +0.38 (+0.10 to +0.62), 46 answers | +0.23 (-0.06 to +0.51), 47 answers | +0.33 (+0.07 to +0.56), 48 answers |
| Two or more readable pages, by readable pages | +0.03 (-0.25 to +0.32), 46 answers | +0.11 (-0.30 to +0.49), 22 answers | +0.15 (-0.17 to +0.46), 38 answers | +0.23 (-0.06 to +0.51), 47 answers | +0.33 (+0.07 to +0.56), 48 answers |
| One or more cited pages, by cited pages | -0.10 (-0.35 to +0.18), 46 answers | +0.27 (-0.01 to +0.53), 43 answers | +0.37 (+0.10 to +0.61), 49 answers | +0.22 (-0.05 to +0.45), 47 answers | +0.13 (-0.14 to +0.39), 48 answers |
| Two or more cited pages, by cited pages | -0.10 (-0.35 to +0.18), 46 answers | +0.14 (-0.22 to +0.46), 29 answers | +0.26 (-0.06 to +0.54), 42 answers | +0.22 (-0.05 to +0.45), 47 answers | +0.13 (-0.14 to +0.39), 48 answers |
Two or more readable pages, by readable pages: the interval includes zero for Google AI Overviews, ChatGPT, Gemini, Perplexity and does not include zero for Claude (API).
Citing answers by the number of pages they cite: Google AI Overviews 45 of 46 (98%) cite three or more and 1 of 46 (2%) one or two; ChatGPT 11 of 43 (26%) cite three or more and 32 of 43 (74%) one or two; Gemini 28 of 49 (57%) cite three or more and 21 of 49 (43%) one or two; Perplexity 47 of 47 (100%) cite three or more and 0 of 47 (0%) one or two; Claude (API) 48 of 48 (100%) cite three or more and 0 of 48 (0%) one or two.
S2b. What one page contributes: the pair share and the cuts
| View | Google AI Overviews | ChatGPT | Gemini | Perplexity | Claude (API) |
|---|---|---|---|---|---|
| (Figure, readable cited page) pairs in which the page holds the figure, pooled | 1617 of 4235 (38%) | 367 of 832 (44%) | 876 of 1671 (52%) | 3436 of 7613 (45%) | 2280 of 4576 (50%) |
| The same in answers with 1 readable page | no answer | 81 of 171 (47%), 19 answers | 43 of 85 (51%), 8 answers | no answer | no answer |
| The same in answers with 2 readable pages | 62 of 110 (56%), 5 answers | 159 of 330 (48%), 14 answers | 234 of 386 (61%), 17 answers | 3 of 20 (15%), 1 answer | 70 of 102 (69%), 2 answers |
| The same in answers with 3 to 4 readable pages | 280 of 498 (56%), 9 answers | 98 of 231 (42%), 7 answers | 500 of 998 (50%), 18 answers | no answer | 519 of 984 (53%), 16 answers |
| The same in answers with 5 to 7 readable pages | 591 of 1467 (40%), 18 answers | 29 of 100 (29%), 1 answer | 99 of 202 (49%), 3 answers | 242 of 691 (35%), 9 answers | 1691 of 3490 (48%), 30 answers |
| The same in answers with 8 or more readable pages | 684 of 2160 (32%), 14 answers | no answer | no answer | 3191 of 6902 (46%), 37 answers | no answer |
| Each answer cut to 2 of its readable pages at random: expected share found, pooled over figures | 57% | 54% | 63% | 59% | 67% |
| The same, median per answer | 54% | 48% | 66% | 60% | 71% |
| Answers with fewer than 2 readable pages, which this cut leaves as they are | 1 of 47 (2%) | 25 of 47 (53%) | 12 of 50 (24%) | 0 of 47 (0%) | 2 of 50 (4%) |
| Each answer cut to 3 of its readable pages at random: expected share found, pooled over figures | 65% | 56% | 67% | 68% | 76% |
| The same, median per answer | 67% | 55% | 73% | 69% | 83% |
| Answers with fewer than 3 readable pages, which this cut leaves as they are | 6 of 47 (13%) | 39 of 47 (83%) | 29 of 50 (58%) | 1 of 47 (2%) | 4 of 50 (8%) |
| Each answer cut to 1 of its readable pages at random: expected share found, pooled over figures | 42% | 42% | 50% | 44% | 50% |
| The same, median per answer | 34% | 33% | 50% | 42% | 47% |
The pair share falls as lists grow, so it does not compare engines whose lists differ in length. The cut to k pages is an exact expectation: for a figure found on f of its answer's n readable pages, the chance that at least one of k pages drawn without replacement holds it. It is a pooled expectation, not a median of answers, and an answer with fewer than k readable pages keeps all of them. Each engine stands on its own answers here; the cut as a median per answer on the same questions is in table S6.
S3. Placebo: pages cited for questions of other sectors
Each answer's pages are replaced by the same number of readable pages cited for questions of other sectors: an answer draws as many pages as it has readable pages. The placebo share is the exact expectation of the part of the figures the check then finds, over every choice of such pages. The pool, the same for all five engines: the readable pages of the main set, without the pages that Google AI Overviews, ChatGPT or Gemini cite for a question of the answer's sector (327 to 352 pages per sector). Pages of this pool that Perplexity or Claude (API) cite for a question of the answer's sector: 0. Second pool: the readable pages cited by Perplexity or Claude (API), without the pages either of them cites for a question of the answer's sector (476 to 528 pages per sector).
| Figures | Google AI Overviews | ChatGPT | Gemini | Perplexity | Claude (API) |
|---|---|---|---|---|---|
| All figures: found on the cited pages | 507 of 640 (79%) | 267 of 470 (57%) | 453 of 661 (69%) | 644 of 755 (85%) | 822 of 939 (88%) |
| All figures: placebo | 28% | 18% | 20% | 40% | 24% |
| All figures: placebo, second pool | 30% | 19% | 21% | 43% | 25% |
| Figures with a unit: found on the cited pages | 355 of 473 (75%) | 186 of 365 (51%) | 312 of 475 (66%) | 467 of 576 (81%) | 678 of 775 (87%) |
| Figures with a unit: placebo | 19% | 9% | 10% | 33% | 17% |
| Figures with a unit: placebo, second pool | 20% | 10% | 11% | 35% | 19% |
| Figures matched on the number alone: found on the cited pages | 152 of 166 (92%) | 81 of 105 (77%) | 141 of 181 (78%) | 177 of 177 (100%) | 144 of 163 (88%) |
| Figures matched on the number alone: placebo | 55% | 50% | 47% | 66% | 54% |
| Figures matched on the number alone: placebo, second pool | 57% | 51% | 49% | 68% | 55% |
| AI tools: found on the cited pages | 112 of 113 (99%) | 78 of 85 (92%) | 87 of 88 (99%) | 115 of 116 (99%) | 118 of 119 (99%) |
| AI tools: placebo | 42% | 26% | 30% | 61% | 45% |
| Business software: found on the cited pages | 71 of 90 (79%) | 40 of 92 (43%) | 118 of 157 (75%) | 200 of 250 (80%) | 229 of 240 (95%) |
| Business software: placebo | 24% | 16% | 17% | 45% | 27% |
| Consumer electronics: found on the cited pages | 61 of 89 (69%) | 51 of 89 (57%) | 32 of 74 (43%) | 107 of 117 (91%) | 104 of 127 (82%) |
| Consumer electronics: placebo | 22% | 20% | 12% | 40% | 27% |
| Legal and local services: found on the cited pages | 111 of 135 (82%) | 25 of 65 (38%) | 106 of 150 (71%) | 90 of 105 (86%) | 168 of 182 (92%) |
| Legal and local services: placebo | 21% | 10% | 21% | 19% | 16% |
| Personal finance: found on the cited pages | 89 of 122 (73%) | 55 of 84 (65%) | 65 of 120 (54%) | 80 of 98 (82%) | 126 of 162 (78%) |
| Personal finance: placebo | 17% | 14% | 11% | 27% | 10% |
| Travel: found on the cited pages | 63 of 91 (69%) | 18 of 55 (33%) | 45 of 72 (62%) | 52 of 69 (75%) | 77 of 109 (71%) |
| Travel: placebo | 46% | 20% | 35% | 42% | 23% |
| Placebo per answer, median on the 41 questions of table S1 | 22% | 16% | 15% | 34% | 19% |
| Placebo per answer, median on the 41 questions of table S1, second pool | 25% | 18% | 17% | 37% | 21% |
A check of the expectation by 1000 seeded draws per answer, all figures: Google AI Overviews 28.0% drawn against 28.1%, ChatGPT 18.0% drawn against 18.0%, Gemini 20.1% drawn against 19.9%, Perplexity 40.3% drawn against 40.4%, Claude (API) 23.6% drawn against 23.5%. The match that computes the placebo was compared with the check outputs, figure by figure, for all five engines (1694 figures of Perplexity and Claude (API)): no difference.
S4. Units, attached sources, and figures not on a readable page
| View | Google AI Overviews | ChatGPT | Gemini | Perplexity | Claude (API) |
|---|---|---|---|---|---|
| Figures with a unit: found | 355 of 473 (75%) | 186 of 365 (51%) | 312 of 475 (66%) | 467 of 576 (81%) | 678 of 775 (87%) |
| Figures without a unit (matched on the number alone): found | 152 of 166 (92%) | 81 of 105 (77%) | 141 of 181 (78%) | 177 of 177 (100%) | 144 of 163 (88%) |
| Found figures that were matched on the number alone | 152 of 507 (30%) | 81 of 267 (30%) | 141 of 453 (31%) | 177 of 644 (27%) | 144 of 822 (18%) |
| Figures whose sentence has no source attached | 307 of 640 (48%) | 285 of 470 (61%) | 291 of 661 (44%) | not available | 233 of 939 (25%) |
| The same, within the answers that cite a page | 292 of 625 (47%) | 257 of 442 (58%) | 281 of 651 (43%) | not available | 220 of 926 (24%) |
| Where a source is attached: found on it | 202 of 333 (61%) | 127 of 185 (69%) | 281 of 370 (76%) | not available | 574 of 706 (81%) |
| Found on another cited page: sentence has no source attached | 242 of 305 (79%) | 130 of 140 (93%) | 142 of 172 (83%) | not available | 188 of 248 (76%) |
| Not found, in answers that cite a page (not found plus page not readable) | 117 of 640 (18%) | 175 of 470 (37%) | 193 of 661 (29%) | 109 of 755 (14%) | 103 of 939 (11%) |
| Figures of answers that cite no source | 15 of 640 (2%) | 28 of 470 (6%) | 10 of 661 (2%) | 0 of 755 (0%) | 13 of 939 (1%) |
| Figures the check could not read | 1 | 0 | 5 | 2 | 1 |
| Every figure that was not found (the three rows above) | 133 of 640 (21%) | 203 of 470 (43%) | 208 of 661 (31%) | 111 of 755 (15%) | 117 of 939 (12%) |
| Answers with figures that cite at least one unreadable page | 28 of 47 (60%) | 13 of 47 (28%) | 13 of 50 (26%) | 42 of 47 (89%) | 23 of 50 (46%) |
All five engines: 695 of 2693 (26%) of the found figures were matched on the number alone; 792 of 3456 (23%) of the figures the check could read carry no unit. Google AI Overviews, ChatGPT and Gemini: 374 of 1227 (30%) and 452 of 1765 (26%).
S5. Sector cells: answers, and figures of answers that cite no source
| Sector | Google AI Overviews, answers with figures | Google AI Overviews, figures in answers that cite no source | ChatGPT, answers with figures | ChatGPT, figures in answers that cite no source | Gemini, answers with figures | Gemini, figures in answers that cite no source | Perplexity, answers with figures | Perplexity, figures in answers that cite no source | Claude (API), answers with figures | Claude (API), figures in answers that cite no source |
|---|---|---|---|---|---|---|---|---|---|---|
| AI tools | 8 | 0 of 113 (0%) | 8 | 0 of 85 (0%) | 9 | 0 of 88 (0%) | 8 | 0 of 116 (0%) | 9 | 0 of 119 (0%) |
| Business software | 9 | 0 of 90 (0%) | 8 | 0 of 92 (0%) | 9 | 0 of 157 (0%) | 9 | 0 of 250 (0%) | 9 | 0 of 240 (0%) |
| Consumer electronics | 7 | 15 of 89 (17%) (ELC-16) | 8 | 0 of 89 (0%) | 8 | 10 of 74 (14%) (ELC-13) | 8 | 0 of 117 (0%) | 8 | 0 of 127 (0%) |
| Legal and local services | 8 | 0 of 135 (0%) | 7 | 26 of 65 (40%) (SVC-06, SVC-09, SVC-10) | 8 | 0 of 150 (0%) | 8 | 0 of 105 (0%) | 8 | 0 of 182 (0%) |
| Personal finance | 8 | 0 of 122 (0%) | 8 | 0 of 84 (0%) | 8 | 0 of 120 (0%) | 7 | 0 of 98 (0%) | 8 | 0 of 162 (0%) |
| Travel | 7 | 0 of 91 (0%) | 8 | 2 of 55 (4%) (TRV-08) | 8 | 0 of 72 (0%) | 7 | 0 of 69 (0%) | 8 | 13 of 109 (12%) (TRV-08, TRV-16) |
A sector cell holds 7 to 9 answers with figures.
S6. The like-for-like comparison in four views
Medians per answer with the registered interval, on the same questions for all five engines. The share minus the placebo is the answer's share found minus the mean placebo expectation of its figures (table S3), in percentage points.
| View | Questions | Google AI Overviews | ChatGPT | Gemini | Perplexity | Claude (API) |
|---|---|---|---|---|---|---|
| All figures (table S1) | 41 | 82% (72% to 92%) | 57% (42% to 75%) | 71% (67% to 91%) | 92% (86% to 100%) | 91% (88% to 96%) |
| Figures with a unit only, on the questions where all five answers hold one | 39 | 80% (67% to 91%) | 50% (27% to 67%) | 70% (47% to 90%) | 90% (78% to 100%) | 92% (85% to 100%) |
| Each answer cut to 2 of its readable pages at random: expected share found, median per answer | 41 | 54% (46% to 69%) | 52% (42% to 67%) | 67% (55% to 71%) | 60% (52% to 65%) | 69% (55% to 76%) |
| Share found minus the answer's placebo | 41 | 51 points (44 to 60 points) | 38 points (30 to 50 points) | 59 points (41 to 67 points) | 50 points (38 to 58 points) | 67 points (57 to 70 points) |
| Share found minus the answer's placebo, second pool | 41 | 50 points (40 to 58 points) | 36 points (28 to 51 points) | 59 points (41 to 64 points) | 47 points (37 to 55 points) | 64 points (54 to 68 points) |
All figures. Intervals on the 41 questions. Do not overlap: ChatGPT and Perplexity; ChatGPT and Claude (API). The other 8 of the 10 pairs overlap.
Figures with a unit only, on the questions where all five answers hold one. Intervals on the 39 questions. Do not overlap: ChatGPT and Perplexity; ChatGPT and Claude (API). Meet at one value: Google AI Overviews and ChatGPT. The other 7 of the 10 pairs overlap.
Each answer cut to 2 of its readable pages at random. Intervals on the 41 questions. Do not overlap: no pair. All 10 pairs overlap.
Share found minus the answer's placebo. Intervals on the 41 questions. Do not overlap: ChatGPT and Claude (API). The other 9 of the 10 pairs overlap.
Share found minus the answer's placebo, second pool. Intervals on the 41 questions. Do not overlap: ChatGPT and Claude (API). The other 9 of the 10 pairs overlap.
The cut is an exact expectation over every choice of two pages, and an answer with fewer than two readable pages keeps all of them and is not cut: on the 41 questions, Google AI Overviews 1 of 41, ChatGPT 20 of 41, Gemini 11 of 41, Perplexity 0 of 41, Claude (API) 1 of 41 answers.
Each engine against ChatGPT, the engine with the fewest pages per answer, on the same questions. Each resample draws one list of questions and uses it for both engines. Differences are in percentage points.
| View | Pair | Questions | Statistic | Difference | 95% interval of the difference | First higher | Lower | Equal |
|---|---|---|---|---|---|---|---|---|
| All figures | Google AI Overviews minus ChatGPT | 41 | difference of the medians | +25 points | +8 to +42 points | 27 of 41 | 7 of 41 | 7 of 41 |
| All figures | Gemini minus ChatGPT | 41 | difference of the medians | +14 points | -0 to +36 points | 23 of 41 | 11 of 41 | 7 of 41 |
| All figures | Perplexity minus ChatGPT | 41 | difference of the medians | +35 points | +18 to +50 points | 30 of 41 | 3 of 41 | 8 of 41 |
| All figures | Claude (API) minus ChatGPT | 41 | difference of the medians | +34 points | +16 to +51 points | 30 of 41 | 5 of 41 | 6 of 41 |
| Figures with a unit only | Google AI Overviews minus ChatGPT | 39 | difference of the medians | +30 points | +14 to +48 points | 24 of 39 | 7 of 39 | 8 of 39 |
| Figures with a unit only | Gemini minus ChatGPT | 39 | difference of the medians | +20 points | -1 to +43 points | 20 of 39 | 10 of 39 | 9 of 39 |
| Figures with a unit only | Perplexity minus ChatGPT | 39 | difference of the medians | +40 points | +25 to +62 points | 28 of 39 | 5 of 39 | 6 of 39 |
| Figures with a unit only | Claude (API) minus ChatGPT | 39 | difference of the medians | +42 points | +26 to +63 points | 30 of 39 | 3 of 39 | 6 of 39 |
| Each answer cut to 2 of its readable pages | Google AI Overviews minus ChatGPT | 41 | difference of the medians | +2 points | -9 to +17 points | 15 of 41 | 25 of 41 | 1 of 41 |
| Each answer cut to 2 of its readable pages | Gemini minus ChatGPT | 41 | difference of the medians | +14 points | -3 to +25 points | 25 of 41 | 10 of 41 | 6 of 41 |
| Each answer cut to 2 of its readable pages | Perplexity minus ChatGPT | 41 | difference of the medians | +8 points | -6 to +18 points | 18 of 41 | 23 of 41 | 0 of 41 |
| Each answer cut to 2 of its readable pages | Claude (API) minus ChatGPT | 41 | difference of the medians | +17 points | -2 to +32 points | 22 of 41 | 19 of 41 | 0 of 41 |
| Share found minus the answer's placebo | Google AI Overviews minus ChatGPT | 41 | difference of the medians | +13 points | -1 to +28 points | 23 of 41 | 18 of 41 | 0 of 41 |
| Share found minus the answer's placebo | Gemini minus ChatGPT | 41 | difference of the medians | +22 points | -2 to +32 points | 26 of 41 | 14 of 41 | 1 of 41 |
| Share found minus the answer's placebo | Perplexity minus ChatGPT | 41 | difference of the medians | +13 points | -6 to +25 points | 25 of 41 | 16 of 41 | 0 of 41 |
| Share found minus the answer's placebo | Claude (API) minus ChatGPT | 41 | difference of the medians | +29 points | +10 to +38 points | 31 of 41 | 10 of 41 | 0 of 41 |
| Share found minus the answer's placebo, second pool | Google AI Overviews minus ChatGPT | 41 | difference of the medians | +14 points | -2 to +27 points | 22 of 41 | 19 of 41 | 0 of 41 |
| Share found minus the answer's placebo, second pool | Gemini minus ChatGPT | 41 | difference of the medians | +23 points | -2 to +33 points | 26 of 41 | 14 of 41 | 1 of 41 |
| Share found minus the answer's placebo, second pool | Perplexity minus ChatGPT | 41 | difference of the medians | +11 points | -8 to +24 points | 25 of 41 | 16 of 41 | 0 of 41 |
| Share found minus the answer's placebo, second pool | Claude (API) minus ChatGPT | 41 | difference of the medians | +28 points | +7 to +39 points | 32 of 41 | 9 of 41 | 0 of 41 |
| Questions where both answers have two or more readable pages | Google AI Overviews minus ChatGPT | 20 | mean of the differences | +15 points | +6 to +25 points | 12 of 20 | 2 of 20 | 6 of 20 |
| Questions where both answers have two or more readable pages | Gemini minus ChatGPT | 17 | mean of the differences | +7 points | -10 to +23 points | 8 of 17 | 4 of 17 | 5 of 17 |
| Questions where both answers have two or more readable pages | Perplexity minus ChatGPT | 22 | mean of the differences | +13 points | +5 to +22 points | 13 of 22 | 2 of 22 | 7 of 22 |
| Questions where both answers have two or more readable pages | Claude (API) minus ChatGPT | 22 | mean of the differences | +19 points | +9 to +29 points | 14 of 22 | 3 of 22 | 5 of 22 |
Tables of the sources the engines cite
Written from addendum/analysis/source-overlap.json (addendum/analysis/source_overlap.py) and, for the views of all five engines that file does not hold, from the key five_engines.sources of report/side-views.json (report/side_views.py, with the functions of the first script). Page key: host in lower case without www. + path without a trailing slash; query and fragment dropped, except the video id (v) of a youtube.com/watch address. The analysis of sources is exploratory: the preregistration does not contain it (deviation log, entry 3).
A1. Sources per engine
| Engine | Answers | Answers with sources | Page citations | Distinct pages | Distinct domains | Median pages per answer | Answers with sources a marker names and the record does not list | Such sources | Answers that cite YouTube | Distinct YouTube pages |
|---|---|---|---|---|---|---|---|---|---|---|
| Google AI Overviews | 49 | 48 | 357 | 336 | 208 | 7 | 0 | 0 | 21 | 38 |
| ChatGPT | 50 | 44 | 83 | 76 | 60 | 2 | 32 | 52 | 0 | 0 |
| Gemini | 50 | 49 | 136 | 121 | 99 | 3 | 25 | 45 | 2 | 2 |
| Perplexity | 50 | 50 | 600 | 553 | 327 | 10 | not recorded | not recorded | 0 | 0 |
| Claude (API) | 50 | 48 | 272 | 236 | 188 | 6 | not recorded | not recorded | 0 | 0 |
All five engines: 239 of 249 answers cite at least one source, on 1012 distinct pages. The four engines without Claude (API): 191 of 199 answers.
Google AI Overviews, ChatGPT and Gemini: 468 distinct pages by this key. The check of figures counts the same answers' sources as 483 addresses, each address as it was cited, with its query string.
ChatGPT: 6 answers cite no source (BIZ-04, SVC-06, SVC-07, SVC-09, SVC-10, TRV-08); their saved pages hold 0 "Sources" controls and 0 outbound links, which indicates that ChatGPT answered them without a search. In 32 of the other 44 a marker names more sources than the panel listed (52 mentions). Gemini: 25 answers hold markers that name 45 sources the record has no address for.
Lists fully seen (sources cited, and no marker names an unlisted source): ChatGPT 12 answers, Gemini 24, Google AI Overviews 48. ChatGPT's list is fully seen in 11 of the 40 five-engine questions, and both ChatGPT's and Gemini's in 4 of the 40 (four engines: 4 of the 41).
Perplexity: 50 answers; interface language tr (50); account signed in, free plan (50); 5 of 50 records carry the notice that a preview of the advanced search was switched on, and 10 further records note that it could not be observed reliably. The records hold 608 source entries; 8 of them repeat a page of the same answer under the page key, which leaves 600 page citations. Claude (API): model claude-sonnet-5-5 (50); location setting New York, US (50); searches allowed per answer 3; 48 of 50 answers ran one search, 2 ran none, and none ran more: 1 in 48 answers, 0 in 2 answers.
A2. Overlap of the sources cited for the same question
Jaccard over all engines of the row, on the questions where each of them cites at least one source: the mean and the median of the per-question ratios. Pooled: shared sources of all questions over the sources of all questions. A row of 11 questions gives counts only: one question holds its shared page. The row of four engines without Claude (API) is the view comparable with the two earlier rounds of the Cross-Engine Citation Study, which had those four engines.
| Engines | Questions | Mean overlap, pages | Median, pages | Questions with a shared page | Pooled, pages | Mean overlap, domains | Median, domains | Questions with a shared domain | Pooled, domains |
|---|---|---|---|---|---|---|---|---|---|
| All five: Google AI Overviews, ChatGPT, Gemini, Perplexity, Claude (API) | 40 | 0.3% | 0.0% | 2 of 40 | 2 of 925 | 1.0% | 0.0% | 5 of 40 | 5 of 694 |
| All five, only questions whose ChatGPT source list was fully seen | 11 | counts only | 1 of 11 | 1 of 244 | counts only | 1 of 11 | 1 of 198 | ||
| Four without ChatGPT: Google AI Overviews, Gemini, Perplexity, Claude (API) | 45 | 1.2% | 0.0% | 10 of 45 | 12 of 975 | 2.6% | 0.0% | 15 of 45 | 18 of 753 |
| Four without Claude (API): Google AI Overviews, ChatGPT, Gemini, Perplexity | 41 | 0.4% | 0.0% | 3 of 41 | 3 of 825 | 1.6% | 0.0% | 8 of 41 | 8 of 617 |
| The same four, only questions whose ChatGPT source list was fully seen | 11 | counts only | 1 of 11 | 1 of 208 | counts only | 2 of 11 | 2 of 169 | ||
| Google AI Overviews, Gemini, Perplexity | 47 | 3.0% | 0.0% | 21 of 47 | 25 of 867 | 5.1% | 4.5% | 24 of 47 | 32 of 672 |
| Google AI Overviews, Gemini, Perplexity, without Perplexity's question SVC-07 | 46 | 2.9% | 0.0% | 20 of 46 | 24 of 853 | 4.9% | 2.3% | 23 of 46 | 30 of 659 |
| Google AI Overviews, Gemini, Perplexity, only questions whose Gemini source list was fully seen | 22 | 3.9% | 2.4% | 11 of 22 | 14 of 369 | 6.4% | 5.7% | 12 of 22 | 17 of 283 |
| Google AI Overviews, ChatGPT, Gemini | 41 | 0.8% | 0.0% | 3 of 41 | 3 of 452 | 2.9% | 0.0% | 8 of 41 | 8 of 366 |
The five-engine row holds the 41 questions of the four-engine row without TRV-16, for which Claude (API) cites no source. Both rows without Perplexity's question SVC-07: 40 and 41 questions, unchanged (ChatGPT cites no source for that question).
A2b. Sources by the number of engines that cite them
Counted per question: a source cited for two questions is counted twice. All five engines, on the questions of the five-engine row:
| Level | Questions | Sources, counted per question | Cited by one engine | By two | By three | By four | By all five | Questions where at least two engines share a source | At least three | At least four | All five | Distinct sources over these questions |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Pages | 40 | 925 | 733 of 925 (79%) | 135 of 925 (15%) | 40 of 925 (4%) | 15 of 925 (2%) | 2 of 925 | 40 of 40 | 32 of 40 | 15 of 40 | 2 of 40 | 825 |
| Domains | 40 | 694 | 487 of 694 (70%) | 132 of 694 (19%) | 45 of 694 (6%) | 25 of 694 (4%) | 5 of 694 (1%) | 40 of 40 | 36 of 40 | 26 of 40 | 5 of 40 | 456 |
The four engines without Claude (API), on the questions of the four-engine row:
| Level | Questions | Sources, counted per question | Cited by one engine | By two | By three | By all four | Questions where at least two engines share a source | At least three | All four | Distinct sources over these questions |
|---|---|---|---|---|---|---|---|---|---|---|
| Pages | 41 | 825 | 686 of 825 (83%) | 111 of 825 (13%) | 25 of 825 (3%) | 3 of 825 | 41 of 41 | 21 of 41 | 3 of 41 | 745 |
| Domains | 41 | 617 | 453 of 617 (73%) | 123 of 617 (20%) | 33 of 617 (5%) | 8 of 617 (1%) | 41 of 41 | 31 of 41 | 8 of 41 | 408 |
A3. What the list sizes allow: the ceiling of the all-engine overlap
The Jaccard of several lists cannot exceed the shortest list divided by the longest. Means over the questions of the row. An engine has the shortest list when no other list is shorter (ties count for each).
| Row | Questions | Observed mean overlap | Highest mean overlap the list sizes allow | The same, given the observed union | Questions where the shortest list has one source | Shortest list: Google AI Overviews | Shortest list: ChatGPT | Shortest list: Gemini | Shortest list: Perplexity | Shortest list: Claude (API) |
|---|---|---|---|---|---|---|---|---|---|---|
| All five engines, pages | 40 | 0.3% | 15.2% | 7.7% | 19 of 40 | 0 of 40 | 35 of 40 | 15 of 40 | 0 of 40 | 0 of 40 |
| All five engines, domains | 40 | 1.0% | 17.5% | 9.6% | 22 of 40 | 1 of 40 | 36 of 40 | 13 of 40 | 0 of 40 | 0 of 40 |
| Four engines without ChatGPT, pages | 45 | 1.2% | 23.8% | 12.9% | 6 of 45 | 3 of 45 | not in the row | 42 of 45 | 0 of 45 | 2 of 45 |
| Four engines without ChatGPT, domains | 45 | 2.6% | 28.0% | 16.0% | 6 of 45 | 5 of 45 | not in the row | 42 of 45 | 0 of 45 | 5 of 45 |
| Four engines without Claude (API), pages | 41 | 0.4% | 15.1% | 8.8% | 20 of 41 | 0 of 41 | 36 of 41 | 15 of 41 | 0 of 41 | not in the row |
| Four engines without Claude (API), domains | 41 | 1.6% | 17.3% | 11.0% | 23 of 41 | 1 of 41 | 37 of 41 | 13 of 41 | 0 of 41 | not in the row |
| Google AI Overviews, Gemini and Perplexity, pages | 47 | 3.0% | 24.7% | 15.6% | 6 of 47 | 4 of 47 | not in the row | 46 of 47 | 0 of 47 | not in the row |
| Google AI Overviews, Gemini and Perplexity, domains | 47 | 5.1% | 28.6% | 19.1% | 6 of 47 | 6 of 47 | not in the row | 46 of 47 | 0 of 47 | not in the row |
A4. Overlap by pair of engines: Jaccard and containment
All ten pairs of the five engines. Containment: of the sources the first engine cites, the share the second also cites for the same question, pooled over the questions where both cite at least one source. "First" and "second" follow the order of the pair's name.
| Pair | Questions | Mean overlap, pages | Mean overlap, domains | Questions with a shared page | Questions with a shared domain | Pages of the first also cited by the second | Pages of the second also cited by the first | Domains of the first also cited by the second | Domains of the second also cited by the first |
|---|---|---|---|---|---|---|---|---|---|
| Google AI Overviews and ChatGPT | 42 | 4.7% | 14.0% | 15 | 30 | 15 of 320 (5%) | 15 of 77 (19%) | 33 of 271 (12%) | 33 of 72 (46%) |
| Google AI Overviews and Gemini | 47 | 12.1% | 17.9% | 34 | 37 | 49 of 353 (14%) | 49 of 133 (37%) | 60 of 302 (20%) | 60 of 127 (47%) |
| Google AI Overviews and Perplexity | 48 | 14.7% | 22.8% | 43 | 45 | 107 of 357 (30%) | 107 of 566 (19%) | 128 of 306 (42%) | 128 of 452 (28%) |
| Google AI Overviews and Claude (API) | 46 | 14.4% | 20.2% | 36 | 40 | 68 of 341 (20%) | 68 of 259 (26%) | 79 of 290 (27%) | 79 of 237 (33%) |
| ChatGPT and Gemini | 43 | 2.9% | 7.9% | 4 | 9 | 4 of 81 (5%) | 4 of 123 (3%) | 9 of 76 (12%) | 9 of 117 (8%) |
| ChatGPT and Perplexity | 44 | 3.9% | 9.8% | 18 | 31 | 21 of 83 (25%) | 21 of 535 (4%) | 39 of 78 (50%) | 39 of 415 (9%) |
| ChatGPT and Claude (API) | 43 | 4.0% | 8.7% | 11 | 20 | 11 of 82 (13%) | 11 of 243 (5%) | 21 of 77 (27%) | 21 of 222 (9%) |
| Gemini and Perplexity | 49 | 6.2% | 8.8% | 27 | 31 | 39 of 136 (29%) | 39 of 583 (7%) | 45 of 130 (35%) | 45 of 464 (10%) |
| Gemini and Claude (API) | 47 | 10.3% | 12.7% | 25 | 26 | 35 of 129 (27%) | 35 of 266 (13%) | 38 of 124 (31%) | 38 of 244 (16%) |
| Perplexity and Claude (API) | 48 | 10.8% | 17.1% | 35 | 41 | 73 of 580 (13%) | 73 of 272 (27%) | 93 of 457 (20%) | 93 of 249 (37%) |
Mean overlap of two of the five engines, over the ten pairs: pages, lowest 2.9%, highest 14.7%; domains, lowest 7.9%, highest 22.8%.
Containment on one set of questions (the questions where the engine, Google AI Overviews and Perplexity all cite at least one source): ChatGPT, domains, 42 questions: 33 of 72 (46%) also cited by Google AI Overviews, 35 of 72 (49%) by Perplexity; Claude (API), domains, 46 questions: 79 of 237 (33%) also cited by Google AI Overviews, 89 of 237 (38%) by Perplexity; ChatGPT, pages, 42 questions: 15 of 77 (19%) also cited by Google AI Overviews, 20 of 77 (26%) by Perplexity; Claude (API), pages, 46 questions: 68 of 259 (26%) also cited by Google AI Overviews, 72 of 259 (28%) by Perplexity.
A5. Yardstick (Google AI Overviews, ChatGPT and Gemini): the same engine asked the same question twice (pages)
The repeat set, which exists for the three engines the preregistration names: the first ten questions of the asking order (BIZ-04, TRV-05, AIT-04, FIN-01, FIN-02, ELC-07, AIT-17, TRV-06, SVC-08, ELC-16). A row holds the questions where both lists cite at least one page.
| Comparison | Questions | Mean overlap, pages | Median | Questions with a shared page | Questions without a shared page | Pages of the first answer cited again |
|---|---|---|---|---|---|---|
| Google AI Overviews, first and second answer | 7 | 42% | 31% | 7 of 7 | 0 of 7 | 32 of 51 (63%) |
| ChatGPT, first and second answer | 9 | 24% | 0% | 4 of 9 | 5 of 9 | 5 of 17 (29%) |
| Gemini, first and second answer | 10 | 45.3% | 45.0% | 8 of 10 | 2 of 10 | 17 of 29 (59%) |
| Google AI Overviews and ChatGPT, first answers | 7 | 6% | 0% | 3 of 7 | 4 of 7 | not applicable |
| Google AI Overviews and Gemini, first answers | 8 | 8% | 7% | 5 of 8 | 3 of 8 | not applicable |
| ChatGPT and Gemini, first answers | 9 | 0% | 0% | 0 of 9 | 9 of 9 | not applicable |
A mean or median over fewer than 10 questions is given as a whole percentage.
A5b. The yardstick like for like (pages)
For a pair of engines: the questions of the repeat set where both engines cite at least one page in both askings. The two engines' first answers stand beside each engine's own two answers on the same questions.
| Pair (first and second) | Questions | Two engines: mean overlap | Two engines: median | First engine asked twice: mean | First engine asked twice: median | Second engine asked twice: mean | Second engine asked twice: median | Questions where the two engines share less than either shares with itself |
|---|---|---|---|---|---|---|---|---|
| Google AI Overviews and ChatGPT | 6 | 7% | 4% | 47% | 32% | 31% | 17% | 3 of 6 (50%) |
| Google AI Overviews and Gemini | 7 | 8% | 6% | 42% | 31% | 50% | 40% | 5 of 7 (71%) |
| ChatGPT and Gemini | 9 | 0% | 0% | 24% | 0% | 50% | 50% | 3 of 9 (33%) |
Across the three pairs the mean overlap of two engines runs from 0% to 8%, and the mean overlap of one engine asked twice from 24% to 50%.
A6. Most cited domains, five engines
543 distinct domains. Cited by one engine only: 339 (62%); by two: 115; by three: 49; by four: 34; by all five: 6 (bankrate.com, github.blog, kayak.com, samsung.com, tomshardware.com, tsa.gov). Hosts counted under one platform domain: medium.com 2, substack.com 2.
| Domain | Answers citing it | Engines | Google AI Overviews | ChatGPT | Gemini | Perplexity | Claude (API) |
|---|---|---|---|---|---|---|---|
| nerdwallet.com | 37 | 4 | 10 | 4 | 0 | 13 | 10 |
| reddit.com | 29 | 4 | 11 | 1 | 4 | 13 | 0 |
| youtube.com | 23 | 2 | 21 | 0 | 2 | 0 | 0 |
| forbes.com | 15 | 2 | 4 | 0 | 0 | 11 | 0 |
| bankrate.com | 14 | 5 | 3 | 2 | 3 | 3 | 3 |
| midjourney.com | 13 | 4 | 3 | 4 | 0 | 3 | 3 |
| pcmag.com | 13 | 3 | 4 | 0 | 4 | 5 | 0 |
| github.com | 12 | 4 | 3 | 5 | 1 | 3 | 0 |
| angi.com | 11 | 4 | 4 | 1 | 1 | 0 | 5 |
| consumeraffairs.com | 10 | 4 | 2 | 0 | 2 | 2 | 4 |
| tsa.gov | 10 | 5 | 3 | 1 | 2 | 3 | 1 |
| banani.co | 9 | 3 | 2 | 0 | 0 | 4 | 3 |
| eesel.ai | 9 | 4 | 2 | 0 | 1 | 3 | 3 |
| experian.com | 9 | 4 | 2 | 0 | 3 | 3 | 1 |
| homeguide.com | 9 | 4 | 3 | 0 | 1 | 3 | 2 |
| costbench.com | 8 | 2 | 0 | 0 | 0 | 7 | 1 |
| extraspace.com | 8 | 4 | 3 | 0 | 2 | 2 | 1 |
| kayak.com | 8 | 5 | 2 | 1 | 1 | 2 | 2 |
| yahoo.com | 8 | 3 | 2 | 0 | 0 | 4 | 2 |
| bestbuy.com | 7 | 4 | 1 | 1 | 0 | 4 | 1 |
The four engines without Claude (API): 468 distinct domains. Cited by one engine only: 308 (66%); by two: 104; by three: 46; by all four: 10 (atlassian.com, bankrate.com, costloop.app, github.blog, github.com, kayak.com, reddit.com, samsung.com, tomshardware.com, tsa.gov).