How we measure AI answers and why it holds
Our frameworks in words and never the prompts. Each one is versioned and dated so that a reading made today can be compared with one made next year.
The AI survey framework
We ask the AI models the same questions a customer survey asks people and then we compare the two sets of answers on one scale.
- Start with a survey that already exists. Yours or a public one. For UK banks we use the CMA service quality survey which asks customers whether they would recommend their bank and its app and its overdraft and its branches.
- Ask the AI models the survey question about each company in the category. "Would you recommend Northbank as a main current account? Yes or no with the main reason." Every company in the category is asked in the same words.
- Ask each question several times and count the answers that say yes. Five models are asked every question three times with web search off and on. The AI score is the share of answers that say yes and the survey score is the share of customers who would recommend. The two sit on one scale.
- Add the brand measures that matter to you. Trust or value or innovation or the claims you want checked. The same rule applies to each one because every company is asked in the same words.
Why a question is asked more than once
A model does not give the same answer twice. The chart shows one survey question asked ten times of one model and the share that said yes each time. The reading is the average with the range beside it and nothing is reported from a single run.
The journey framework
Every question we ask belongs to one step of the user's journey and every measure is read at the step where it matters.
Working out what matters. Trends behind the answer.
Which brands to consider. AI visibility and associations.
Can this one be trusted. Claim accuracy and brand versus competitor and the perception gap.
Choosing and buying. Cited but not named and the site audit on converting pages.
Using and coming back. Out of scope for now.
The frame is what makes a reading a plan. A brand that is missing at Shortlist has a content problem and a brand that loses at Verify has a facts problem and the two need different teams. The Discovery page shows the analysis at each step.
Proxy and synthetic prompts
Nobody can see the questions people type into an AI model and no model publishes them. What can be seen is what people search for and what they ask on forums and what they tell surveys. We build the prompt set from those sources so that it follows what people are actually asking rather than a list written a year ago. A proxy prompt is a real search question asked of the model in the words people use. A synthetic prompt is one we write to test a claim or a comparison the real questions do not reach. Both are kept with the reading and both carry a version.
The frameworks behind the services
The 30-day routine plan is the framework behind the AI consulting work. It opens with the team's week by task and ends with a count of the routines still in use at day 30 and it is described on the AI consulting page.
The three-layer content plan is the framework behind the content strategy work. It reads the pages behind the answers for what the brand is known for and where its pages are present at each step and whether each page is ready to be read and it is described on the AI brand content strategy page.
The agent method agrees one job in a sentence and writes the instructions in plain English and runs the agent beside the person who does the job today. It is described on the AI agents page.
Versions
Every framework and every prompt set carries a version and a date and a change is recorded so that two readings can always be compared on the same terms or told apart when the method moved.
Questions people ask us
Why ask an AI model a survey question?
Because it puts the AI's view and people's view on one scale. A survey asks whether people would recommend a bank and we ask the models the same question in the same words and the gap between the two is the finding.
Does Model Research publish its prompts?
No. The frameworks are published and versioned so that the method can be examined and the prompts are not because the prompts are the instrument. A client sees every prompt that produced their reading.
Can the method read a new AI model?
Yes. The method is AI and platform agnostic and a new model is a new column in the record. The same questions are asked in the same words and the counting is done outside the model.
Questions about the method are welcome. Our details are on the contact page.