AI Index / Methodology

How the Index is measured

The AI Recommendation Index measures what AI assistants actually recommend when buyers ask. Every number on an Index page traces back to the process below — and every page discloses its own sample size and window. The measurement parameters on this page (models, gates, guardrails) are read live from the pipeline that collects the data, not hand-maintained copy.

§ Prompt design

Each category carries a fixed set of buyer-shaped prompts — 6–8 by design; the Statistical robustness section reads the live distribution: the questions real buyers type, like “best [category] for [audience]”, “[category] alternatives”, or “what should I use for [job]”.

Prompts are strictly neutral — they never contain product names, never steer toward or away from any vendor, and stay stable between samples so trends compare like with like.

§ Prompt provenance

Where prompts come from: each category’s set is written from its slot in the category taxonomy plus kit-derived buyer questions — the questions a buyer in that category actually types into an engine — and each prompt is tagged with the buying intent it represents (best-in-category, alternatives, problem-solving, …). Intent tags are published with the per-intent breakdowns on category pages.

Every prompt is hashed and versioned. Each stored run carries a stable hash of the exact prompt wording it answered, stamped alongside the extraction version that parsed it. The hash stays with the run forever, so any published number can be traced to the precise wording that produced it.

Editing prompts mid-series: prompt wording is frozen while a series runs. When a prompt does have to change (ambiguity, category drift), the new wording gets a new hash — historical runs keep the hash of the wording they actually answered, per-prompt analyses group by hash, and the reworded prompt starts a new series instead of being silently spliced into its predecessor’s. Nothing is relabeled after the fact.

§ Engines & measured models

Categories are measured across up to 5 engines via their official APIs, with web search enabled where the engine supports it. An AI engine is not one thing — answers depend on the exact model behind it, so the Index discloses the model measured per engine:

  • CHATGPT (OpenAI) — model: chat-latest · changeover in progress — published numbers count only chat-latest runs
  • CLAUDE (Anthropic) — model: claude-sonnet-5 · changeover in progress — published numbers count only claude-sonnet-5 runs
  • GEMINI (Google) — model: gemini-2.5-flash
  • PERPLEXITY (Perplexity) — model: sonar
  • GROK (xAI) — model: grok-4.20-0309-non-reasoning

Each category page shows per-engine shares separately — engines disagree, and that disagreement is part of the data. Not every category runs on every engine; the engines measured are listed per category.

§ Sampling, windows & the minimum-sample gate

Sampling runs continuously, spread across the batch cycle to avoid time-of-day artifacts. At the current measurement budget, each engine samples each category roughly monthly — cadence scales with budget, and every published number discloses exactly how many sampled answers it rests on.

Published numbers aggregate a 28-day (4-week) rolling window; trends compare that window against the 28 days before it.

Categories with fewer than 20 sampled answers in the current window are marked “insufficient data” and publish no rankings at all. Just above that floor the confidence interval is very wide — read the interval, not the point estimate, and treat the ranking as provisional until the sample grows.

recommendation_share = % of sampled answers in the window that recommend the product · avg position = mean rank when answers are ordered lists

§ Sampling design

AI answers are stochastic — the same prompt to the same engine can return a different list — so published rates are estimates over repeated samples, not single draws. Each prompt-engine pair is sampled per cycle — read live from the configured sampling parameters. If the weekly budget projection is exceeded mid-cycle the runner degrades the rest of that cycle to single draws, so realized sampling can be lower than the value shown here. Per engine:

  • CHATGPT — samples per prompt per cycle: 1
  • CLAUDE — samples per prompt per cycle: 1
  • GEMINI — samples per prompt per cycle: 1
  • PERPLEXITY — samples per prompt per cycle: 1
  • GROK — samples per prompt per cycle: 1

An engine’s N can be lower than the global value when its per-run cost would multiply the most expensive line first; a global value of 1 means repeated sampling is switched off and every number is a single draw — whatever is running is what this section shows. Samples from one prompt-engine cycle are tagged as one sample group, so they are never double-submitted and never mistaken for independent prompts. Sample sizes on category pages count answers analyzed, so repeated samples legitimately increase them. During ramp-up, only the top 50 categories by exposure sample above 1× — every other category runs single draws until the sampling budget is proven.

Cadence and rotation: sampling runs continuously under an engine-aware rotation — categories are prioritized by their most-behind engine, and an engine that already covered a category inside the current freshness window is skipped so calls go to the laggards, not to keeping already-fresh pairs artificially fresh. At the current budget each engine reaches each category roughly monthly; cost caps are enforced before submission, and when a cycle’s projected cost would exceed the remaining budget the runner degrades to single draws for the remainder instead of silently overspending or dropping coverage.

Production-default parameters, on purpose: samples are issued with each engine’s production-default request parameters — no temperature tuning, no parameter sweeps. We measure the answer distribution real users actually get, not a distribution tuned to be convenient to measure; two samples differ only because the engine’s own randomness differs.

§ Statistical robustness

The Index is a sample, not a census — so here are the questions a statistically literate reader should ask before trusting a number on it, answered from the live corpus where the answer is measurable and answered “not yet” where it isn’t.

How many prompts per category?

1–8 prompts per active category (median 7), across 298 categories with a written prompt set. Prompts are neutral, intent-tagged, and stable between samples — see Prompt design.

How many independent runs per category?

Each run is one sampled answer: one prompt, one engine, one call, no conversational context carried between them. Runs are not all independent, though: each prompt-engine pair is sampled several times per cycle, so those runs share a single draw of the prompt. We report total answers analyzed; every statistical gate — the publishing gate below and the confidence intervals — counts independent prompt executions instead, so repeated sampling can never buy a category a narrower interval or an easier path past the gate. In the current 28-day window, categories carry 2–72 runs (median 28)287 categories sampled, 235 of them at or above the 20-run publishing gate. Every category page states its own sample size; nothing is published off an undisclosed n.

Which engines, and which models?

Up to 5 engines via their official APIs, and the exact model measured per engine is read live from the collection pipeline and listed in Engines & measured models. Model changeovers are logged and never spliced into one series — the models are part of the measurement, so a change to them is a change to the instrument.

How is variance reported?

Every ranked entry publishes a 95% Wilson score confidence interval on its recommendation share, computed on that entry’s mention-runs over the category’s sample size. Wilson rather than the normal approximation because shares here routinely sit near zero on small samples, where the normal interval leaks below 0 and undercovers.

One honest caveat, and one correction to it. Repeated samples of the same prompt are clustered, not independent, so counting them as separate observations would narrow every interval by roughly the square root of the sample count — we correct for that by computing the interval over independent prompt executions rather than raw answers. What we do not correct for: the prompt set itself is fixed, so the interval describes uncertainty given these questions, not uncertainty about which questions buyers ask. True uncertainty is therefore still somewhat wider than the stated band. Prompt randomization is on the not-yet-measured list on this page.

p = x/n · center = (p + z²/2n) / (1 + z²/n) · half = z·√( p(1−p)/n + z²/4n² ) / (1 + z²/n) · z = 1.96

Category pages render the interval next to the share, so two entries whose intervals overlap can be read as what they are: not distinguishable at this sample size. Each entry also publishes a first-pick rate — the share of its mention-runs where it led the answer’s list — because being named at all and being the top recommendation are different claims. Trends carry a separate comparability gate (see Trend guardrails).

Are prompts randomized or paraphrased?

Not yet. Prompt wording is fixed per category, which buys clean trend comparability — the same question asked over time — at the cost of measuring only that phrasing. A different phrasing of the same buying question can surface a different set of products, and the Index currently cannot tell you how much. Rotating a randomized paraphrase set alongside the fixed set, so phrasing sensitivity is itself measured, is on the roadmap.

Are personas simulated?

No. Prompts carry the audience wording a category’s buyers use (“for freelancers”, “for small teams”), but there are no simulated user profiles, no memory, and no account history behind a run. What an engine recommends to a logged-in user it has learned from is outside what the Index measures.

Is there geographic variation?

Not measured. Runs are issued from a single region without locale hints, so every published number is one geography’s view. Engines can and do localize recommendations; the Index does not currently quantify that, and no number on it should be read as a global average.

What this measurement cannot tell you

AI answers are probabilistic. The same prompt to the same model on the same day can return a different list, which is exactly why nothing here rests on a single answer and why every share ships with an interval instead of a decimal that looks more certain than it is. Engines are also a moving target: vendors change models, retrieval and grounding behavior shift without announcement, and a share can move because the instrument moved rather than because the market did. The guardrails above exist to keep that from being published as a finding.

So: the Index measures what a set of engines recommended, for a fixed set of neutral prompts, in one region, over a 28-day window — not what “AI thinks” and not what any individual user will see. Where a number is too thin to defend, we publish an em-dash instead. The live statistics behind this section are on /data, and if any of it doesn’t hold up we would rather hear it: hello@orbator.io.

§ Model changeovers

Engine vendors retire and replace models. When the measured model behind an engine changes, the Index never splices the two into one series: published aggregation is restricted to the new model’s runs, and trends for that engine are suppressed or flagged until the new model has accumulated a comparable window. The models in the Engines section above are read live and always show what is measured right now.

DateEngineChangeStatus
Jul 21, 2026CLAUDE (Anthropic)claude-haiku-4-5 claude-sonnet-5announced — takes effect when the environment override lands
Jul 21, 2026CHATGPT (OpenAI)gpt-4o-mini chat-latestannounced — takes effect when the environment override lands

§ Entity resolution

Every product mentioned in an answer is extracted and resolved against a canonical registry, so “HubSpot”, “Hubspot CRM” and “hubspot.com” count as one entity. Resolution matches by domain first, then by name and known aliases; cited URLs serve as a confirmation signal.

Generic phrases (“CRM software”, “the tool”) are filtered and never become entities. Newly seen entities and suspected duplicates are auto-flagged for human review; confirmed duplicates are merged with their history re-pointed, and confirmed junk is removed from the rankings.

§ Parsing rules

Mention detection. A product counts as mentioned only when the answer names it or cites its domain. Names match on whole-word boundaries against the registry’s canonical names and aliases — never substrings — and names shorter than three characters are never matched in prose (their domain still counts). Dictionary-word product names additionally require corroboration — exact cased spelling, a ranked-list position, or the product’s own domain cited — per the ambiguous-name rule in Junk handling.

Position. When an answer is an ordered list, each mentioned product’s rank in it is recorded; average position and the first-pick rate derive from those ranks, and answers that aren’t lists contribute mentions without positions.

Mention classes. Every mention in every run is classified four ways, and the class decides what it may count toward:

  • Recommended — the answer explicitly recommends the product. Counts toward recommendation share.
  • Listed — placed on a ranked or bare list without an explicit endorsement. Counts toward recommendation share: being placed on the list a buyer asked for is a recommendation surface.
  • Neutral mention — appears in passing (context, comparison, integration: “works with Salesforce”). Stored, published in the breakdown, never counted as a recommendation.
  • Warned against — the answer advises avoiding it. Never counted as a recommendation, anywhere downstream.

The full four-way breakdown is published per entry, so a share can always be decomposed into what produced it. In effect since extraction v6 (Aug 11, 2026) — see the Methodology changelog for the dated rule history.

§ Junk handling & corrections

Word frequency is not a recommendation, so the pipeline enforces rules that keep junk out of published shares:

  • Ambiguous-name corroboration. Products named after dictionary words (ChatBot, Writer, Wave) count only when the sighting is corroborated — the exact cased spelling, a position in a ranked list, or the product’s own domain cited in the answer. “Any chatbot platform” never becomes a sighting of ChatBot.
  • Recommendations, not mentions. A product counts toward recommendation share only when the answer actually recommends it — a ranked-list position or a recommendation cue. Context mentions (“works for Amazon sellers”, “integrates with Salesforce”) are stored but excluded from published shares.
  • Human review & rejection. Newly seen entities and suspected duplicates queue for admin review; confirmed duplicates are merged and confirmed junk is rejected — rejected entities never appear in rankings, and an automated daily integrity check flags any that somehow do.

See something wrong — a misresolved product, a junk entity, a number that doesn’t hold up? Corrections: hello@orbator.io. We would rather pull a number than publish a wrong one.

§ The archive & what we publish from it

Every day, each published category’s ranking is frozen as a dated, append-only record. Nothing in that archive is ever edited: a correction appends a new revision alongside the original, with a note saying what changed, so the record of the record survives. Each entry discloses whether it was captured live that day or reconstructed later from stored runs, and which extraction rules produced the sightings underneath it.

One policy decides what may leave the archive, and every public page and API endpoint serving answer content is routed through it — the rules are enforced in one place rather than restated in each surface:

  • Index answers only. Customer scans are the customer’s data and never appear here, at any granularity — not as quotes, not as counts.
  • Short excerpts, never full answers. The quotable unit is at most 400 characters of what an engine said. Complete engine output is retained internally for re-extraction and audit, and is never served publicly.
  • Per-engine quoting rules. Some engines’ terms discourage redistributing generated text. Those engines are recorded as having answered, on the date they answered, with the excerpt withheld — the measurement is ours to publish, the prose is not.
  • Intent, not prompts. The buying intent behind a question is published (“best in category”, “alternatives”); the prompt text itself is not.
  • No internals. Costs, token usage and classification internals are never part of a public view. The published shape is built from an explicit list of fields, so a new column cannot leak into it by default.

Retention: the runs and citations that make up the archive are kept indefinitely — they are the record. Full answer texts are retained for at least 24 months and then reviewed.

§ Methodology changelog

The rules that turn answers into published numbers — extraction filters, corroboration requirements, metric definitions, sampling design — improve as we find and fix measurement bugs. Every change is disclosed here, dated, in one changelog: series breaks and metric redefinitions are announced, not discovered. Extraction-rule changes bump the extraction version; every stored sighting is stamped with the version that produced it, and a CI guard fails any version bump that lacks a dated entry below — this changelog cannot silently rot. Current extraction version: 8.

Sep 14, 2026extraction-v8
Directory and marketplace exclusion applies to local business scans

Directories and lead marketplaces are no longer reported as competitors when a local business is measured. Until now the aggregator filter only ran on categories the Index itself had flagged as local, so a scan started from a customer's own product — which carries no Index category — kept listing Houzz, Angi, Thumbtack or HomeAdvisor alongside real rivals. The filter now follows the business, not the category, and the taxonomy gained the home-services aggregators it was missing (Houzz, Angi/Angie's List, HomeAdvisor, Thumbtack, Porch, Nextdoor, BBB).

affects: recommendation share · ranked entities · local categories · entity extraction
Aug 11, 2026extraction-v7
Stored sightings carry the 4-way role; counted basis redefined

Stored sightings carry the 4-way classification as their role. Recommendation share counts explicit recommendations and bare ranked-list placements (being placed on a list is a recommendation surface); warnings ("avoid X") and passing context mentions never count. The full mention breakdown is published per entry.

affects: recommendation share · mention classes
Aug 11, 2026extraction-v6
4-way mention classification recorded per run

Every run now records a 4-way mention classification — recommended, listed, neutral mention, or warned against — so a warning or a passing mention can never be counted as a recommendation anywhere downstream.

affects: mention classes · recommendation share
Repeated sampling, volatility scores, and error bars

Published rates become estimates over repeated samples instead of single stochastic draws: N samples per prompt-engine per cycle (cost-capped, top categories first), per-entry volatility (within-cycle disagreement + across-cycle movement) computed in aggregates, and ±error bars with stability badges in the product dashboard. The methodology page gains sampling design, prompt provenance, parsing rules, and 'what we cannot see' sections, plus this changelog.

affects: sample sizes · confidence intervals · volatility · dashboard · methodology page
Aug 8, 2026extraction-v5
Consumer platform taxonomy

Consumer platforms and marketplaces (Yelp, TripAdvisor, OpenTable, DoorDash, …) excluded as recommendations in software categories — they are sources answers cite, not products — except in the categories where they are the ranked product (e.g. OpenTable in reservation software). Platform domains can no longer be captured by, or credit sightings to, a named entity ("Yelp API").

affects: recommendation share · ranked entities · local categories
Wilson confidence intervals published per entry

Every ranked entry now publishes a 95% Wilson score confidence interval on its recommendation share, computed over the category's sample size — two entries whose intervals overlap are not distinguishable at that sample size, and the pages say so.

affects: confidence intervals · category pages
Jul 21, 2026extraction-v4
Ambiguous-name corroboration; context mentions stop counting

Dictionary-word product names (ChatBot, Writer, Wave) require corroboration — exact spelling, a ranked-list position, or their own domain cited — and context-only mentions ("works for Amazon sellers") no longer count toward recommendation share.

affects: recommendation share · entity extraction
Jul 21, 2026extraction-v3
AI assistant products re-admitted where they belong

AI assistant products (ChatGPT, GitHub Copilot, …) re-admitted as valid entities — only in categories that rank assistant products, where they belong on the leaderboard.

affects: ranked entities · assistant categories
Jul 17, 2026extraction-v2
Exclusions enforced at the storage chokepoint

The v2 source/platform exclusions are enforced at the single storage chokepoint, closing paths where an excluded source could still be stored as a sighting.

affects: recommendation share · ranked entities
Jul 9, 2026extraction-v2
Source and platform exclusions

Review sites, analyst firms, social platforms, and search infrastructure excluded from rankings — they are where answers look things up, not recommendations.

affects: recommendation share · ranked entities
Jun 11, 2026extraction-v1
Index launch — original extraction rules

Entity extraction from AI answers ships with the Index: products named in answers are resolved against the canonical registry, generic phrases are filtered, no source/platform filters yet.

affects: recommendation share · entity extraction

§ What we cannot see

A methodology that only lists what it measures is marketing. These are the things this method cannot observe — actual blind spots, not caveats:

  • Logged-in personalization and memory. Runs are clean API calls: no account history, no memory, no custom instructions. What an engine recommends to a user it knows is invisible to us — and can differ systematically from what it tells a stranger.
  • Geographic and account-level variance. Runs issue from one region with no locale hints. Engines localize answers; we do not measure by how much. Every published number is one geography’s view, not a global average.
  • Model-side A/B tests and silent model swaps. Vendors run server-side experiments and swap model builds without announcement. We disclose the model identifier we call, but the identifier is not the build — a share can move because the instrument moved. Announced changeovers get the era handling above; silent ones we cannot detect except as unexplained volatility.
  • Engines we do not sample. The Index measures the engines listed above, through their APIs. Google AI Overviews, Microsoft Copilot, Meta AI, voice assistants, and in-product surfaces (the ChatGPT web app with browsing, custom GPTs) are not measured — and API answers can differ from what the same vendor’s consumer surface says.
  • Perplexity answer text. Licensing limits the retention and redistribution of Perplexity’s raw answer text, so raw engine output stays internal (re-extraction and audit) and only derived metrics — shares, positions, mention classes — are published. Third parties cannot re-parse our Perplexity corpus themselves; the published extraction rules and version history are the audit trail we can offer for it.
  • Residual stochastic error. Repeated samples bound stochastic error; they do not eliminate it. The confidence intervals and volatility scores published with every number are the honest size of what remains — a rate without them would look more certain than it is.

The Statistical robustness section answers the related questions (randomization, personas, geography) with the same rule: an explicit “not yet” over a vague reassurance, every time.

§ Independence

Orbator customers are badged on Index pages for disclosure. Customer status does not affect measurement — the prompts, sampling schedule, extraction, and ranking math are identical for every product, customer or not. Rankings cannot be bought, and no one can pay to be removed.

The Index exists because we sell AI-visibility tooling — that is exactly why the measurement itself has to be, and is, untouched by who pays us.

§ Using the data

Every category is downloadable as CSV from its page and queryable via the free JSON API (/api/index/categories; the measurement parameters on this page at /api/index/methodology; docs at /developers). Free to use with attribution to orbator.io. Questions or corrections: hello@orbator.io.

[ORBATOR]

© 2026 Orbator. All rights reserved.