Why are five providers five different answers?
We measure through the official APIs of five AI providers: ChatGPT (OpenAI), Claude (Anthropic), Gemini (Google), Perplexity, Grok (xAI). Those are five different models, each trained on its own mix of text, each with its own tuning and its own way of consulting the web. The same customer question therefore produces five different answers, with partly different names and partly different arguments. Assistants we do not measure stay out of the report; what is not measured is never presented as a result.
There is another difference you cannot see: every model has its own knowledge boundary. In its models overview, Anthropic defines the “reliable knowledge cutoff” as “the date through which the model's knowledge is most extensive and reliable”. Two providers can therefore literally hold a different version of your market in memory, simply because their training stopped on a different date.
Why ask the same question five times in a row?
Because one answer is not repeatable. Anthropic's API documentation states it literally: results are “even with temperature of 0.0 … not fully deterministic”. Ask the same question twice in a row and you can get two different lists of names, without anything having changed about your business or about the question. Anyone who concludes from a single answer that they are “not in there” is really measuring chance.
Five repetitions per question are not statistical certainty — we do not pretend otherwise. They are enough to tell “never named” apart from “sometimes named”, and to show a spread instead of a single lucky hit. That is why we report, per customer question, how often you appeared in five attempts, per provider, rather than one summarised figure.
What is the difference between memory and web search?
An assistant can answer in two ways. From the model's memory: then it draws on the text it was trained on, and you see what has been written about you over the past years. Or with web search: then it fetches pages while answering and uses those. Those are two different questions with two different answers, and adding them together hides exactly what you want to know. A business that is only named through web search stands somewhere else than a business that also comes out of memory — the first depends on that moment's search, the second sits in the model itself.
So we measure those two lanes separately, where a provider's API allows it. Where it does not, the report states which lane was measured and which was not measured. What you can do with either outcome is in “AI doesn't mention you — what to do?”.
Why does your own ChatGPT app differ from an API measurement?
What you see in your own app is your answer — not the answer. Consumer apps take your conversation history, your location, your language, your settings and sometimes a different model version than the API into account. Typing in your own business and seeing yourself come back proves nothing: the assistant knows you just searched for yourself. That is not a fault of the app; it is exactly what it was built for.
An API measurement deliberately does none of that. No history, no profile, a frozen question, a frozen model, a frozen date — the same for everyone and repeatable by us. An API measurement is therefore not a better picture of your own experience, but it is the only one you can compare with a month from now.
What does one screenshot prove, and what do 125 answers prove?
| What varies | How much | Why it is counted separately |
|---|---|---|
| Provider | 5: ChatGPT (OpenAI), Claude (Anthropic), Gemini (Google), Perplexity, Grok (xAI) | Different model, different source mix, different knowledge boundary |
| Customer question | 5, frozen before the measurement | One question is too narrow; five cover how customers really ask |
| Repetition per question | 5 | The same model does not answer the same twice in a row |
| Lane | 2, where the API allows it: from memory and with web search | Two different questions; added together they hide each other |
| Total | 5 × 5 × 5 = 125 planned answers | Enough to see a pattern, not enough to predict the future |
A screenshot is one cell from that table: at one of the five providers, on one question, in one attempt, in one lane, at one moment. It can be accurate and still mean nothing. A hundred and twenty-five answers are no proof of what happens tomorrow, but they do show whether you are never, sometimes or often named, on which question that tips, and at which provider. Every answer gets an evidence ID, so you can look up the literal sentence instead of taking our word for it.
The full setup, including how the questions are frozen and how the evidence IDs work, is on our measurement method.