In part one, I proposed four questions that help determine whether an agency really knows what it plans to do and why.
If you have not read part 1 yet:
In part 2, we go one step further.
A good explanation of the working method is not yet proof of a result.
Today, it is not enough to ask: “Do you have case studies and numbers?” Many companies already have numbers.
A much more useful question is: “Can I understand exactly how this result was calculated — and can it be fairly compared with the starting point?”
I would check four things:
- exactly how the result was measured;
- what the case study actually shows;
- how much involvement the business owner will need to provide;
- what the agency can genuinely guarantee.
A number only makes sense together with the method used to calculate it
Suppose an agency presents a case study: “After three months, the client’s AI visibility increased to 80%.” It sounds convincing.
But today I would immediately ask:
- which questions were used;
- how many there were;
- which AI systems were checked;
- in which market and language;
- over what period;
- whether measurements were repeated;
- whether “before” and “after” were compared using comparable methodology.
The number 80% alone does not explain what was actually measured.
If the initial result was calculated using one set of questions and the later result using another, the attractive “before and after” is difficult to interpret as a change in company visibility.
My basic question is therefore: “Please show me exactly how this metric was calculated.”
At KAN, the initial and subsequent measurements use comparable methodology. Otherwise, the difference between “before” and “after” cannot fairly be interpreted as a change in visibility.
This matters because AI answers are not completely stable in themselves.
In a 2026 study, researchers compared Google Search, AI Overviews and Gemini using a benchmark of 11,500 user queries. One conclusion was that AI Overviews were less stable when the same query was repeated and were sensitive even to small changes in wording.
That is why one favourable AI answer is not enough to draw conclusions about a company’s visibility.
One attractive result in ChatGPT is not yet a case study
If I see a screenshot saying, “ChatGPT recommends our client in first place,” that is an interesting fact. But it is not yet evidence of a lasting result.
I want to know:
- the exact question that was asked;
- why that question was chosen;
- which other questions were also checked;
- over what period;
- what the situation was before the work began;
- what happened in other AI systems.
One screenshot proves only that, in that specific situation, AI named the company.
A good case study looks different to me: starting point → work period → comparable follow-up measurement → result.
It is also worth seeing the results for individual AI systems separately.
We observed this in KAN’s own initial measurement of the Polish market: in the same commercial niche, the most visible companies differed between ChatGPT, Perplexity, Gemini and Google AI.
So instead of asking, “Do you have successful case studies?”, I would ask: “Can I see the initial and later results calculated using the same methodology?”
That shows much more clearly what really stands behind an attractive number.
The business owner should know in advance how much time they will need to contribute
The third question is: “How much of my time will the cooperation require, and exactly what will you need my knowledge for?”
The promise “We do not need anything from you; we will do everything ourselves” does not seem like an advantage to me.
Technical work can indeed be delegated. But a provider cannot know on its own:
- which situations clients encounter regularly;
- how expert decisions are made;
- which questions clients ask most often;
- which real cases demonstrate the company’s distinctive experience;
- which figures actually appear inside the business;
- where the important professional nuances lie.
Without this, it is easy to create correct but interchangeable content.
In the methodology behind KAN’s work, the boundary is simple: the technical work can be delegated, but the meaning and business knowledge must come from the owner or a genuine expert.
Before signing a contract, I therefore want to know why the agency will need my involvement and how much time it will really require.
A good provider should neither promise fully expert work entirely “without the client” nor shift delivery of the whole service onto the client.
For me, the right model is: the agency organises and carries out the process, while the owner supplies knowledge that cannot credibly be invented on their behalf.
A guarantee of first place in ChatGPT is a red flag
The final question is: “What exactly can you guarantee?”
An agency can guarantee what remains under its control:
- scope of work;
- deadlines;
- preparation of agreed materials;
- regularity of measurements;
- reporting format;
- delivery of the agreed plan.
But if I hear “In three months ChatGPT will recommend your company in first place” or “We guarantee that your brand will be added to ChatGPT’s memory,” that is a red flag.
The reason is simple: the provider does not control the final AI answer.
OpenAI’s official documentation explains the conditions that help content remain available to ChatGPT Search, but content availability is not a guarantee that a page will appear in a particular answer.
Still less can anyone guarantee manually “adding” a company to a model’s memory.
I have much more confidence in the statement: “We work to increase the likelihood that AI can find, correctly understand and more often recommend your company, and we will measure the changes.” Than in: “We will secure first place for you in ChatGPT.”
Numbers alone do not mean that a result has been proven
After analysing the market, I would no longer simply ask, “Do you have case studies?”
A much more interesting question is whether I can understand how the result shown was achieved.
Which questions? Which AI systems? Which market and language? What starting point? What period? Are the “before” and “after” measurements comparable?
These details turn “the client’s result increased to 80%” from an attractive number into a result that can genuinely be assessed.
When these questions are not enough
These criteria are useful when a provider sells work on a company’s visibility and recommendations in AI.
If you only need SEO, website development, advertising or standard content marketing, the selection criteria will be different.
Nor do I expect an agency to disclose its entire internal technology, platform list or production process before a contract is signed.
I need a different level of transparency: I want to understand what I am buying, how the result will be assessed and where the agency’s area of control ends.
Two parts — two levels of verification
In part one, the main question was: “Does the agency understand what it plans to do and why?”
In part two: “Can it show the result in a way that lets me understand how it was calculated?”
For me, the combination of these two things distinguishes an attractive presentation from an understandable working system.
Return to part 1:
Sources
- Grossman R. et al. — How Generative AI Disrupts Search: An Empirical Study of Google Search, Gemini, and AI Overviews
- OpenAI — Publishers and Developers FAQ
Not sure how visible your company is in AI today?
At KAN, we start with an initial measurement. We check whether the company appears in ChatGPT, Gemini, Perplexity and Google AI answers, which companies are recommended instead, and where the biggest gap lies.
Only then do we decide what is worth changing.
BOOK A CALL