AI visibility is volatile by design: the same question can produce a different answer, with different sources and different brands, a few days later. Google's AI Overviews have a 70% chance of changing between one observation and the next, and nearly half (45.5%) of the cited sources are new each time (Ahrefs, 43,000+ keywords, November 2025). That means a single check of what AI says about your brand is a sample of one, not a measurement.
This piece covers why AI answers move, how much they move, and what a measurement approach that survives the movement looks like.
How often do AI answers actually change?
More often than most teams assume. Three independent sources point the same way.
Google's AI Overviews. Ahrefs tracked more than 43,000 keywords, each with at least 16 recorded AI Overviews over a month. An AI Overview lasted an average of 2.15 days before its content changed, and the checks were not daily, so Ahrefs notes the true rate is likely higher. Between consecutive responses, only 54.5% of cited URLs carried over. Named entities, which include brands, stayed consistent 54% of the time (Ahrefs, November 2025).
Google's two AI surfaces disagree with each other. For the same query, AI Mode and AI Overviews cited the same URLs only 13.7% of the time, across 540,000 query pairs of US data from September 2025 (Ahrefs). The answers meant roughly the same thing, with 86% average semantic similarity, but they credited different pages. If you only check one surface, you only know about one surface.
The models themselves are not deterministic. Researchers ran five language models configured to be "deterministic" on eight tasks over 10 runs and saw accuracy vary by up to 15% across runs (Atil et al., "Non-Determinism of 'Deterministic' LLM Settings", arXiv, revised April 2025). Separately, Thinking Machines Lab sent the same prompt to an open model 1,000 times at temperature zero and got 80 unique completions (September 2025). The first 102 tokens matched; then the answers split.
Why does the meaning stay stable while the sources churn?
Here is the part that matters for brands. Ahrefs found an average cosine similarity of 0.95 between consecutive AI Overviews, which it described as very consistent meaning, even while almost half the sources rotated. AI engines tend to keep the same story and swap the evidence underneath it.
That has two consequences.
Narrative is sticky. If an engine has settled on a description of your category, and your brand is not part of it, a single new article rarely changes the story overnight. You need repeated, consistent signals across many sources.
Citations are slippery. Being cited on Monday says little about Thursday. A page that wins a citation can lose it to a near-identical page from a competitor or a publisher in the next refresh, with no change in quality on either side.
So the stable thing to measure is whether you are part of the answer over time, not whether you appeared in one screenshot.
Why do one-off checks mislead?
A single check fails in predictable ways.
It overstates wins. You test a prompt, your brand appears, and the team celebrates. If the brand only appears in a third of runs, that celebration was luck.
It overstates losses. The reverse is just as common. A brand missing from one answer may be present in most others. Reacting to one bad screenshot sends teams rewriting pages that were working.
It hides engine differences. With a 13.7% citation overlap between AI Mode and AI Overviews, a result on one Google surface tells you little about the other, let alone about ChatGPT, Claude or Perplexity. Each engine needs its own reading. Our guide to how AI search engines choose sources explains why their source pools differ.
It mixes up modes. An engine answering from its training data behaves differently from the same engine searching the web. We cover that split in Memory vs Live. A one-off check rarely records which mode produced the answer.
It cannot show direction. The question leadership asks is "are we gaining or losing?" A single point has no slope.
What does a measurement that survives volatility look like?
Treat AI visibility the way analysts treat any noisy signal: sample repeatedly, store the raw data, and report the trend rather than the latest value.
1. Fix the prompt set. Pick the 20 to 100 questions your buyers actually ask and keep them stable. Changing prompts every month makes trends meaningless. Add new prompts in a separate group so the core set stays comparable.
2. Poll on a schedule. Run the same prompts on a regular cadence, daily or weekly depending on how fast your market moves. Given that AI Overviews changed roughly every two days in the Ahrefs data, monthly checks will miss most of the movement. More readings per period mean a more reliable rate.
3. Report rates, not appearances. "Mentioned in 7 of 10 runs this week" is a measurement. "Mentioned today" is an anecdote. Visibility score and share of voice work because they are rates computed over many answers and compared against competitors.