Same question, different answer
We asked ChatGPT 20 buying questions, three fresh times each. Asked again, the same question kept about 71% of its brands but only 57% of the websites it cited. The recommendations are fairly steady. The sources behind them are not.
Checked Sept 23, 2026
What we did
On Sept 23, 2026 we asked ChatGPT 20 questions a buyer might ask, from "What is the best CRM for a small business?" to "What is the best robot vacuum for pet hair?". Each question went in three times, each time in a new logged-out browser with no history, using ChatGPT's default settings. That gave us 60 answers. For each one we listed the brands named and the websites cited.
To compare two answers to the same question we used overlap: the share of names that appear in both, out of all the names in either. 100% means identical. 0% means nothing in common.
What we found
| Brands | Cited websites | |
|---|---|---|
| Overlap between two runs of the same question | 71% | 57% |
| Appeared in every run of their question | 61 of 128 | 41 of 110 |
Out of 128 brand mentions across all runs, 61 showed up every single time the question was asked. For cited websites it was 41 out of 110. Most sources were one-offs.
By category
| Category | Brand overlap | Source overlap |
|---|---|---|
| AI visibility tools | 100% | 55% |
| Website builders | 93% | 60% |
| VPNs | 93% | 61% |
| CRM software | 87% | 54% |
| Email marketing platforms | 87% | 59% |
| Project management software | 86% | 64% |
| Meal kits | 75% | 45% |
| Running shoes | 38% | 51% |
| Robot vacuums | 36% | 60% |
| Mattresses | 19% | 61% |
What it means
- One screenshot proves nothing. A brand can be missing from one answer and present in the next. Judge visibility by how often you appear across several runs.
- The top of the list is steady. The brands named every time are the real winners in a category. The tail changes run to run.
- Sources churn much more than brands. ChatGPT reaches similar conclusions from different pages each time. If you want to know which pages feed the answer, sample many runs, not one.
- Ask any tool how often it asks. A tracker that runs each question once a day shows you noise as if it were movement.
Limits
This is one day and three runs per question, logged out, from the US, at default settings. Logged-in users with memory, other countries and other thinking settings will differ. We will repeat this weekly and report how it moves.
Common questions
Does ChatGPT give the same answer every time?
No. When we asked the same buying question again in a fresh session, the answer kept about 71% of the brands and 57% of the cited websites.
How many times should you ask ChatGPT to measure visibility?
At least three times per question. One answer can show a brand as present or missing by chance. The brands named every time are the ones that matter.