This page is the complete rulebook for how AIBUY measures AI shopping assistants. Every published number can be traced to prompts, dates, model versions, and a logged answer. When our method changes, the version changes — and old reports keep their original version label.
A structured set of 100+ purchase-intent prompts per category, phrased the way real buyers ask — not the way marketers write. Full prompt sets ship with published reports and client deliverables.
Every prompt goes to every assistant in the panel for that market — one independent call per prompt (no shared session, no memory) — answers captured verbatim with timestamps and model version.
Answers are parsed into brand mentions, positions, stated specs and prices. Scoring rules are fixed and applied identically to every brand in the panel.
A person re-reads a sample of raw answers against the scored output, and every number that reaches a headline is human-checked. Discrepancies fix the rule, not the data.
Charts and drafts are prepared to a fixed editorial standard; conclusions and opinions are written by a person. Every report opens with the verification card stating exactly which parts were human-checked.
Defined once, used everywhere. All four are computed per brand, per assistant, per test window.
| Metric | Question it answers | Definition |
|---|---|---|
| Mention Rate | How often does AI name you at all? | Share of prompts where the brand appears in the answer. Also reported as top-3 mention rate. |
| Rank Position | How early does AI name you? | Average position of first mention in ordered lists (1 = first recommended). Lower is better. |
| Info Accuracy | Does AI describe you correctly? | Share of stated facts (price, key specs, availability) that match the merchant's current official data. Errors are logged verbatim. |
| Suppression Ratio | Who wins your questions? | For prompts where your brand is mentioned, how often a named competitor appears earlier. Reported as a multiple (e.g. 2.4× = the leader appears first 2.4× more often). |
The US-market panel covers the first four; the China-market panel covers the rest. The cross-market project runs identical prompts through both panels. Panels are reviewed quarterly and aligned to officially available API endpoints — we don't list models we can't test.
Today's test surface is each assistant's official API endpoint: one independent call per prompt, no shared session, no memory or personalization, no web retrieval by default; model version and timestamp are logged per answer.
Known boundaries: consumer apps (which do have retrieval and memory) may answer differently; Yuanbao is proxied by its base model Hunyuan; Wenxin and Taobao Ask have no public API. A consumer-surface (web) channel and a public surface-calibration study are in progress. Changing the test surface counts as a method change and bumps the version.
Every run is archived: prompts, verbatim answers, assistant identity, model version, timestamp, and the scoring output. Published reports link to a public excerpt; the full evidence archive stays under our control and is never published wholesale — but every run archive and every public release carries a content fingerprint (hash), so anyone can verify that it has not been altered (a fingerprint proves integrity only, not truth).
Assistants personalize, A/B test and change without notice — our numbers are a structured sample of a moving target, not a census. We state sample sizes and dates everywhere, we don't smooth over week-to-week noise, and we never project GMV impact from mention data.
Method v1.0 (this document) governs all 2026 Q3–Q4 reports. Material changes bump the version and are listed on this page with dates. Historical reports are never re-scored under a new method silently — if we re-run them, both versions are shown.
Cite us: attribute "AIBUY" with a link to the report page, and note the report version, test window and status (pilot/final). Don't alter the numbers, and don't imply partnership or endorsement.
Re-verify: the method, metric definitions and scoring rules are all public on this page; integrity of public releases is checkable via the hash line on each report (offline tool). Spotted a discrepancy? Bring evidence — corrections get a public changelog entry.