Head to Head: who the engines pick when a buyer asks A or B. October 2026.

Abstract

From the September 2026 Absence Index deposit (DOI 10.5281/zenodo.22907695; verbatim answers from ChatGPT, Claude, Perplexity, Google AI Overviews and Google AI Mode, three repeats per question), we read every answer to a head to head question ("A vs B") and every answer to an alternatives question ("alternatives to X") across 14 categories. We report how often an answer declares a single winner, who wins, whether the five engines agree, whether the winner is the more-named vendor in the Index, and what the winning answers cite.

75 of 359 scored head to head answers named one vendor as the better choice for the buyer as asked: 20.9% (95 percent Wilson interval 17.0% to 25.4%). The most common verdict was both, a pick for each of two buyer types: 271 of 359. All five engines picked the same winner on 1 of 24 questions.

Data and question set

The question rule: from the deposit's release record, every target question whose text is an "A vs B" comparison (it contains " vs " or " versus ") and every target question that asks for alternatives to a named vendor ("alternatives to X"). That gives 24 head to head questions and 36 alternatives questions in 12 of the deposit's 14 categories; CRM and project management have no question of either kind. Each question was asked of five engines three times: 360 head to head answers and 540 alternatives answers. After the deposit's exclusion records, 359 and 540 are scored, and every count here is on scored answers only. The answers read here were captured on 22 September 2026.

How the answers were read

The coding rule: An answer declares a winner when its own recommendation sentence names one of the two vendors as the better choice for the question as asked. "Depends" with no pick is coded depends; a pick for each of two buyer types is coded both; a refusal or an answer naming neither is coded neither. The first-named vendor is the first of the two to appear in the text. Cited domains are classed as A's own site, B's own site, a third-party page, or none. Every answer was read twice; disagreements between the two reads were resolved by a third read and logged.

Both passes, the third reads and a consistency read were made by instances of one AI model (Claude) under the written rule, each pass blind to the other; they are not human coders, so agreement between the passes is not evidence that people would code the same way. The head to head passes disagreed on 11 of 360 answers. A precedent fixed at the third read (a pick tied to a condition the question does not state is a split, coded both) was applied to every agreed winner code with a condition in its deciding sentence, and 22 codes changed. 23 alternatives entries went to a third read. Every code, both passes and every change are in the dataset.

The winner rate per engine and overall

Overall: 75 of 359, 20.9% (95 percent Wilson interval 17.0% to 25.4%). The one neither is a request-limit message returned as the answer. Repeats of one question are correlated, so true uncertainty is wider than the interval shown.

The winner table by category

Counts per question, out of its scored answers. Index named count: how many of the category's Index answers named each vendor in the September release.

Agreement

Across repeats: all three repeats of a question on one engine gave the same verdict on 89 of 119 question and engine pairs, and the same single winner on 13 of 119.

Across engines, taking each engine's majority verdict over its three repeats: all five picked the same winner on 1 of 24 questions and gave the same verdict of any kind on 12 of 24. At least one engine's majority named a winner on 11 of 24 questions; on 1 of 24, one engine's majority named A while another's named B.

Winner against the Index named count

Where an answer declared a winner and both vendors carry an Index record with different named counts, the winner was the vendor the Index found named more often in 46 of 70 answers. 3 winning answers are left out because one vendor is outside the Index's checked set for that category, and 2 because the counts are equal. The answer named the question's first vendor before the second in 256 of 359 scored answers; among answers with a winner, the winner was the first-named vendor in 69 of 75.

What winning answers cite

Of 75 answers that declared a winner, 45 of 75 cited the winner's own site, 31 of 75 the rival's own site, 63 of 75 at least one third-party page, and 0 of 75 nothing. By citation: 68 links to the winner's site, 51 to the rival's and 624 to third-party pages. The ten most-cited third-party domains are listed in the report.

Alternatives: first-named alternatives and incumbents

For each "alternatives to X" question: the alternative named first most often, how many different vendors were named first, and how often X itself was named beyond a restatement of the question. The most common first alternative of its question accounted for 384 of 540 scored answers; on 9 of 36 questions every answer led with the same alternative. X was named in 524 of 540 answers.

What the study does not prove

The deposit records answers observed on 22 and 23 September 2026 under method v1.1. A verdict is the answer's wording on that date, not a judgment of product quality. Repeats of one question are correlated, so true uncertainty is wider than the interval shown. A vendor absent from a question's checked set was not measured.

The questions are the Index's own wording, one or two per category, so the findings stand for those questions only, not for every way a buyer might ask.

Dispute route

The Index's own policy applies: any named vendor may ask for one same-configuration re-run per release at no charge, and both results are published. Write to index@broadcastwell.com.

Files and DOI

The report (PDF), the question set, one row per answer with both reading passes, the coding rule, the reader brief, the third-read log and the scripts are on GitHub and Zenodo under CC BY 4.0: DOI 10.5281/zenodo.23202939. Source deposit: DOI 10.5281/zenodo.22907695.