AI Search Visibility Tracking Explained: How to Measure Whether Chatbots Mention Your Brand

What Is AI Search Visibility Tracking?

AI search visibility tracking means asking chatbots the same set of questions on a schedule and recording whether your brand is mentioned, cited as a source, or recommended, plus how it is described. You are measuring frequency across many runs, not looking up a single position, because an AI answer is generated fresh each time.

Think of it as polling rather than looking something up. A rank tracker checks one page for one query and returns one number. A visibility tracker samples a moving target, so the output is a rate: "we appeared in 34 percent of runs for these 40 questions."

That framing changes what counts as a good result. Going from 20 percent to 35 percent across a well-built prompt set is a real signal. Seeing your brand in one answer on a Tuesday is a story you tell yourself.

Why Classic Rank Tracking Doesn't Carry Over

Search results pages are fairly stable. Two people in the same city searching the same phrase see broadly the same ten links. Chatbot answers are built differently: a model reads a prompt, may run several background searches, pulls passages, and writes new text. Small changes anywhere in that chain change the output.

A widely shared 2025 study from SparkToro gathered thousands of prompt runs from hundreds of volunteers and found that asking an AI tool the same brand-recommendation question repeatedly almost never produced the same list twice: fewer than 1 in 100 runs matched. The order of the brands was even less consistent. The study is not peer reviewed and used a limited set of prompts, so treat the exact figures with care. The direction of the finding is what matters, and it matches what anyone sees when they refresh an answer a few times.

Two consequences follow. First, a single answer proves almost nothing, in either direction. Second, the percentage of runs in which you appear was more stable in that research than your position within any one list, so "how often" is a better metric than "how high." If you want the deeper mechanics behind the variation, see why AI search answers can change even when your website has not.

Which Numbers to Track

Start with three measurements. More than that and the spreadsheet becomes a hobby.

Mention and citation are different things. A chatbot can name your company from training data without linking anywhere, and it can cite your article without ever saying your brand name. Tracking both shows which kind of visibility you have. Citation patterns get their own treatment in our guide to AI citation monitoring.

Description quality is the metric people skip and later regret. Being mentioned with an outdated price, a discontinued product or the wrong founding story is a problem, and you only find it by reading the answers. The follow-up guide on finding incorrect descriptions of your company in AI answers covers what to do about it.

How to Build a Prompt Set That Reflects Real Buyers

Your prompt set decides whether the whole exercise means anything. If every prompt contains your brand name, you are measuring whether the model recognizes your name, which tells you little about discovery.

Build 25 to 50 prompts across four groups:

• Category questions. "What are the best options for X?" with no brand named.
• Problem questions. "How do I fix Y?" where your product or content is one plausible answer.
• Comparison questions. "A versus B" including at least one competitor pairing you are part of.
• Brand questions. "What is [your brand] known for?" to check description quality.

Write prompts the way customers type them, including the messy ones. Pull wording from support tickets, sales calls, search query reports and forum threads. Keep a fixed version of the list, because changing the prompts between rounds ruins the comparison. If you must add prompts, add them as a new group and track them separately.

Include the geography and language your customers use. A prompt about "accountants near me" and one about "accountants in Leeds" can surface different answers, so decide up front which you are testing.

How Many Times Should You Run Each Prompt?

Enough times that the percentage stops jumping around. For a quick read, 10 runs per prompt is workable. For anything you plan to report to a boss or a client, 20 or more per prompt gives steadier numbers, and you should say how many runs sit behind each figure.

Run at similar times of day where you can, use fresh sessions rather than one long conversation, and log the date. Personalization matters too. Logged-in accounts with memory turned on can shape answers, so test with a clean profile, and note whether the product had web search enabled, since an answer with live retrieval and one without it are different beasts.

A useful habit is to save the full text of every answer, not just the yes or no. When a number moves, the saved answers let you see why: a competitor started appearing, a new source showed up, or the wording about your product changed.

A Simple Spreadsheet Method

You can run a credible test with nothing but a spreadsheet:

1. List your prompts in column A, grouped by type.
2. For each platform you care about, add columns for run number, mentioned (Y/N), cited (Y/N), cited URL, and a description note.
3. Run each prompt the agreed number of times and paste the answer text into a notes column or a linked document.
4. Use a pivot table to turn Y/N columns into percentages per prompt group and per platform.
5. Repeat on the same schedule and chart the trend.

It is slow, and that slowness is the reason people move to tools. But doing it by hand at least once teaches you what the answers actually look like, which no dashboard can replace. If you are testing across several assistants at once, the companion guide on tracking visibility across ChatGPT, Gemini, Claude and Perplexity explains how their behavior differs.

When a Dedicated Tool Is Worth Paying For

Several commercial products now automate this: they run your prompts on a schedule against multiple AI platforms, store the answers, and chart mention and citation rates. Established SEO suites have added AI visibility modules too. They earn their fee when you have hundreds of prompts, several markets, or stakeholders who want a dashboard.

Before paying, ask a few plain questions. How many runs per prompt does the tool perform, and can you see the raw answers? Does it query the consumer product or an API version, which can behave differently? How does it handle location and login state? A tool that cannot show you the underlying answers is asking you to take its percentages on faith.

Also use the free first-party data that exists. Bing Webmaster Tools began previewing an AI Performance report in February 2026 that shows how often your pages are cited in Copilot and Bing's AI summaries, along with the queries that led to those citations. It reports citations rather than clicks, so it complements your own testing instead of replacing it. Google has said that pages appearing in its AI features are counted within overall Search Console performance data, so there is no separate AI filter to lean on for now.

Reading the Numbers Without Fooling Yourself

Some habits keep the analysis honest:

• Report ranges, not points. "Between 28 and 40 percent" is more truthful than "34 percent" when your sample is small.
• Compare like with like. Same prompts, same platform, same settings, same number of runs.
• Separate your changes from the platform's. Models and search features update without notice. If every brand in your category moved, it was probably the platform.
• Tie it to outcomes. Visibility is a leading indicator. Whether it produces visits or sales shows up elsewhere, which is the subject of our guide to AI search referral analytics.

Frequently Asked Questions