The market filled up fast. Two years ago almost nobody sold generative engine optimization, and now every other search shop has an AI page. Some of them do the work. Many rebranded a slide deck. If you are evaluating an AI SEO agency, the fastest way to tell the difference is to ask questions a rebranded shop cannot answer well. Below are the ones that consistently separate substance from hype, and what a real answer sounds like.
“How do you measure AI visibility, exactly?”
This is the first filter, because it exposes whether they actually track AI search or just talk about it.
A weak answer stays vague, more visibility, better presence, stronger authority. A real answer gets specific: they measure mentions and citations inside named platforms, they track it per platform rather than as one blended number, and they can show you what a report looks like. The per-platform detail matters because these systems disagree with each other. A brand strong in one assistant is often thin in another, so a single combined score hides the gaps that actually cost you.
If an agency cannot describe its measurement method concretely, everything downstream is guesswork.
“What will you change first, and why that?”
Good agencies sequence the work. They do not start everywhere at once.
Listen for a first move that is cheap and high-leverage, usually fixing clarity on your own commercial pages so a model can describe you accurately. That is the correct starting point because you fully control it and it costs the least. If the pitch opens with a large content package or a link-building retainer, be cautious. The evidence does not support volume as the lever. Ahrefs’ August 2025 study of 75,000 brands found the number of pages on a site had almost no relationship with AI visibility, while branded web mentions correlated far more strongly, 0.664 versus 0.218 for backlinks. Ahrefs is clear that this is correlation rather than proven cause, but it still argues against a publish-more strategy.
“Where does your data on AI citation actually come from?”
This question surfaces whether an agency understands the field or is improvising.
You want to hear references to real sources: correlation studies, controlled experiments, platform documentation where it exists. An agency that has done its homework will happily explain that much of what we know is associative, not confirmed ranking logic, and will not oversell certainty. Overconfidence is a tell. Anyone claiming they have cracked exactly how ChatGPT or Google’s AI Overviews rank sources is describing something the platforms themselves have not published.
“Can you show results, and how do you talk about them?”
Case studies are useful, but how an agency frames them tells you more than the numbers.
A trustworthy agency attributes results to its own documented reporting, qualifies them as dependent on market and competition, and does not promise you the same outcome. NotionX, as an example of the honest version, reports client gains in AI mentions and cites specific accounts, while noting those results vary and are not guaranteed. Compare that to a pitch full of round, universal promises. If every prospect supposedly gets a fixed percentage lift on a fixed timeline, the numbers are marketing, not measurement.
Ask specifically whether the reported results come from their own tracking or from a tool anyone can run. Both can be legitimate, but you want to know which.
“What tools do you use, and could I run some myself?”
A confident agency is not threatened by this. The measurement layer is increasingly accessible, and a good partner will tell you plainly what is DIY-able and what genuinely needs them.
This is also a useful area to educate yourself before any sales call. Understanding the landscape of tracking and analysis tools, including the best chatgpt seo software available now, lets you judge whether an agency’s stack is credible and whether their pricing matches the value they add on top of tools you could partly operate yourself. The real value of a strong agency is usually in the ongoing corroboration and monitoring work, not in access to software.
“What happens after we start ranking, do you maintain it?”
Many engagements quietly end at the win. That is a problem, because a citation you earn can slip as competitors publish and models re-evaluate.
A real agency treats monitoring and defense as part of the job, not an upsell. Ask how often they check positions, what triggers a response, and who owns the maintenance. If the answer is fuzzy, you are buying a one-time fix dressed up as a program.
Putting the Answers Together
You do not need every answer to be perfect. You need the pattern to be right: specific measurement, sensible sequencing, evidence-based reasoning, honest framing of results, and a plan for the long game.
Before you even book calls, do one thing that will sharpen every conversation. Ask your top buying questions inside two assistants and save what comes back. Walking into a pitch already knowing where you are visible and where you are missing changes the dynamic entirely. You stop being sold to and start evaluating whether the agency can close a gap you have already seen for yourself.
