Why would you want to check this yourself?
More and more people put their question to an AI assistant instead of a search engine: “who can help me with…”. The answer usually contains a few names and a reason for each. Whether your name is among them, you cannot see anywhere: there are no visitor statistics for a conversation someone has with ChatGPT. The only thing you can do is ask that question yourself and see what comes back. That takes a quarter of an hour and gives a first impression — no more, but no less either.
Which questions do you ask the AI assistant?
Use questions a customer would literally type, in customer language. Not your own trade jargon (“GEO optimisation provider”), but the question as asked by someone who does not yet know what your profession is called. Pick three to five, spread across what you offer and where you work.
| Situation | Question as a customer asks it | What you learn |
|---|---|---|
| Local service provider | “Which accountant in Leiden is good for a sole trader?” | Whether you get named in your region, and next to whom |
| National product | “Which supplier of solar panels for a flat roof is reliable?” | Which arguments the AI uses to choose |
| Specialist | “Who can help me object to my property tax valuation?” | Whether the AI recognises your specialism |
| Comparison | “What is the difference between [your business] and [competitor]?” | How the AI describes you when you are named |
How do you run the self-test step by step?
- Open an AI assistant in a new, empty conversation. Preferably use a window where you are not logged in, or at least a conversation without earlier context about your business.
- Ask the first customer question literally, without mentioning your business name. Note which businesses are named, in what order, and what reason the AI gives for each.
- Start a new conversation and ask exactly the same question twice more. Note again. Do the answers differ? That is normal — more on that below.
- Repeat this for every question. Put everything in a simple table: question, attempt, businesses named, arguments.
- Only then look at the pattern: which names come back in most attempts? Which arguments recur? Where are you — or are you missing?
Why do you get a different answer every time?
A language model does not look things up in a fixed list. It composes every answer anew, word by word, with a built-in dose of chance. That is not a fault but a design choice. In the documentation of the Claude API, Anthropic writes about the setting that controls this chance, literally: “Note that even with temperature of 0.0, the results will not be fully deterministic” — even at the lowest setting the result is not fully predictable. That page is technical documentation for developers (not research, no date shown; we opened it on 1 September 2026), but it is a statement by the vendor itself about its own product.
On top of that, a model has a knowledge boundary. Anthropic calls it, in its models overview, the “reliable knowledge cutoff”: “the date through which the model's knowledge is most extensive and reliable”. Whatever was published about your business after that date, the model only knows if it searches the live web — and whether it does differs per assistant and per setting. Two reasons, then, why a single answer is a snapshot.
What may you conclude from a self-test — and what not?
| What you see | What it does tell you | What it does not tell you |
|---|---|---|
| You are named in 0 of 3 attempts | Right now, with this question, you do not come up spontaneously | Not that you are “unfindable”: another phrasing or another model may turn out differently |
| You are named 1 out of 3 times | You are within the range of possible answers | Not how often that happens on average — three attempts is too few for a percentage |
| A competitor is named every time, with the same reason | A strong signal that this name and argument are firmly anchored | Not why — that requires looking into the sources the model knows |
| The answer refers to an outdated fact about your business | The model knows that fact from older sources, or is reading them live now | Not that it says so everywhere; check which source the AI cites, if any |
In short: a self-test is good for noticing that something is going on. It is not suitable for proving how big it is, or for saying after a month whether anything changed. The numbers are too small and the circumstances too variable for that (logged in or not, which assistant, which day).
When is a real measurement worth it?
A real measurement does three things differently. It repeats each question more often (in our case: five customer questions, five times each, so 125 answers), so that chance averages out and a pattern becomes visible. It records every answer together with the exact request and the exact response, so that every conclusion is traceable. And it keeps the questions frozen, so that a later re-measurement is a fair comparison. That is worth it when you want to decide on the basis of the outcome — for instance which three things to tackle first — or when you want to be able to show whether something changes. Why an AI names only a few businesses at all is, by the way, less certain than it seems; we wrote a separate article in question form about that. Why five providers give five different answers, and why you ask the same question five times, is in “Do all AI assistants say the same?”.