When Is AI Research Good Enough to Act On?
AI can now produce something that looks remarkably like research in minutes.
Ask it to map an audience, explain why Gen Z behaves differently, analyse a category, identify unmet needs, create personas, predict objections to a proposition or tell you what affluent consumers want, and the output can be impressively sophisticated.
This is already changing who produces research.
Consultants and strategists are using AI to build recommendations. Marketing and innovation teams are conducting their own audience analysis. CEOs are interrogating markets directly. Agencies are using AI to develop briefs and cultural territories. Newsletter writers and independent analysts can investigate complex subjects at a speed that once required a research team.
Much of this is genuinely useful.
Some of it may be perfectly sufficient.
The new problem is knowing when it isn’t.
Because the biggest risk of AI-generated research is not necessarily that it is obviously wrong.
It is that it can be plausible, polished and incomplete at exactly the same time.
The new research risk is false confidence
We call this the false confidence problem: research appears rigorous enough to support a decision without actually reducing the uncertainty underneath it.
AI makes this problem harder to spot because it can create coherent arguments, confident synthesis and apparently comprehensive outputs extraordinarily quickly.
The danger is subtle.
The answer may not be false. It may simply be generic, decontextualised, based on weak evidence or missing the human forces that will ultimately determine whether a decision works.
During one of our projects, for example, we asked leading AI tools who was influencing affluent consumers in France.
Napoleon appeared near the top of the list.
It was funny, but also surprisingly revealing.
Stretch the idea of influence far enough and perhaps Napoleon does still matter culturally. But that was clearly not the intelligence required to make a contemporary marketing decision.
The distinction is between something being plausible and something being current, useful and decision-relevant.
That distinction becomes even more important when we move from factual questions to questions about people.
Take Gen Z.
Ask an AI to describe them and you will probably receive a convincing synthesis: digitally native, socially conscious, authenticity-seeking, purpose-driven consumers.
None of those statements is necessarily false.
But the lived reality is considerably messier.
Many young people are economically constrained, sceptical of institutions and reassessing conventional relationships between consumption, identity and status. The interesting strategic question is not whether Gen Z values authenticity. It is how those conditions are changing what authenticity, status, ownership and consumption actually mean.
The difference matters.
A plausible description of an audience is not necessarily enough to decide what to build for them.
We see the same pattern elsewhere.
Parents may understand the practical value of a product and still reject it because of judgement, guilt or what choosing it says about them as a parent.
An affluent audience can have significant wealth while behaving nothing like a conventional “luxury consumer”, carefully mixing everyday value purchases with highly selective splurges.
A brand can have substantial market presence while being largely absent from the cultural environments where its future customers are forming meaning, trust and preference.
These are not necessarily facts that sit neatly inside a dataset waiting to be summarised.
They have to be discovered and interpreted.
The Human Reality Gap
We call the distance between what AI can plausibly infer about people and what we have enough evidence to know about their lived reality the Human Reality Gap.
Sometimes that gap is very small.
AI is extraordinarily useful for desk research, exploration, summarising established knowledge, interrogating large bodies of information, hypothesis generation, brainstorming and pattern spotting within good data.
We have been combining AI and human research at Trybes for around a decade. We are not interested in protecting research from AI. Quite the opposite!
AI can make good research faster, broader and significantly easier to use.
The problem begins when AI synthesis is mistaken for human evidence.
And the size of that problem depends on the decision.
If you are exploring an unfamiliar market before a meeting, AI may be perfectly sufficient.
If you are generating initial hypotheses for a campaign, it may be exactly the right tool.
If you are signing off a major investment, repositioning a brand, designing a new service or making claims about how people will actually behave, the evidential standard should be very different.
The question is therefore not:
Can AI do research?
A more useful question is:
Is the evidence underneath this answer strong enough for the decision we are about to make?
Did we actually learn anything new about people?
There is another very practical way to test AI-assisted research:
Did we actually learn anything new?
Recently, we reviewed a substantial body of work exploring the future of management.
The material identified a series of familiar but credible tensions. Managers are exhausted. They feel insecure. They are under increasing pressure. Technology is changing their roles. They need new capabilities to navigate uncertainty.
None of that was wrong.
But as we moved from the executive summary into the longer material, something unusual happened.
Normally, when we conduct qualitative or ethnographic research, our problem is compression.
There are too many nuances, contradictions, exceptions, tensions and competing truths. The difficult part is reducing all that complexity without destroying what makes it meaningful.
Here, we experienced almost the opposite.
The summary contained most of what appeared in the longer material.
There was more content, but no additional complexity.
That became a methodological warning signal.
Because “managers feel insecure” is not yet an insight.
Research should push further.
Haven’t managers always felt insecure?
If so:
What has changed about the nature of that insecurity now?
Why are the management models that worked before no longer sufficient?
What emotional, professional and personal price are people willing to pay to remain managers?
What compromises will they still make?
What has become non-negotiable?
What would make management an attractive role again?
And then a much bigger question emerged.
The original proposition was essentially about helping managers become better equipped for the future of management.
But what if the problem was not primarily the managers?
What if asking already exhausted people to become more resilient, adaptable and capable simply placed another responsibility on them?
The question changed from:
How should managers adapt to the future of management?
to:
What needs to change around managers for management to become a role people can genuinely thrive in again?
That is a materially different research question.
It could produce a different proposition, different product and different strategy.
And this is one of the clearest differences between synthesis and research.
Synthesis organises what is already visible in the material.
Good research can reveal that you were asking the wrong question in the first place.
What ethnography adds
This distinction is particularly important when terms such as ethnography and netnography are used.
Netnography is not simply collecting online comments and asking an AI to identify recurring themes.
The researcher has to decide what constitutes the field.
Which spaces matter?
Which communities, interactions or behaviours belong inside the investigation?
What is absent?
Who is speaking?
Who is not?
What changes from one environment to another?
What contradictions appear over time?
What assumptions is the researcher bringing into the interpretation?
Ethnographic work involves immersion, context, field notes, reflexivity and interpretation.
The researcher moves backwards and forwards between evidence and questions.
They notice when behaviour conflicts with what someone says.
When different people use the same language to mean very different things.
When a tiny observation destabilises a large assumption.
When something that seemed peripheral becomes central.
When silence, discomfort or contradiction carries more meaning than the polished answer.
AI can contribute to this work. We use it for exploration, desk research, hypothesis generation, interrogation, pattern spotting and working through rigorous bodies of data.
But an AI-generated synthesis of online material randomly chosen by AI does not automatically become netnography because the source material came from the internet.
Nor does a sophisticated synthesis of interviews automatically constitute deep qualitative research.
The methodological distinction matters because research is not only about processing answers.
It is about creating the conditions in which better questions, unexpected evidence and competing interpretations can emerge.
Six questions to ask before trusting AI research
Before acting on AI-generated or heavily AI-assisted research, we believe six questions matter.
1. What is it actually based on?
Where did the evidence come from?
Is the system interpreting real research, reliable sources and observed behaviour, or constructing an answer primarily from its broader training and available information (therefore largely based on marketing articles)?
2. Does the field match the claim?
If the work claims to describe an audience, market or culture, has it actually examined the people and environments relevant to that claim?
A large quantity of material does not compensate for the wrong sample or the wrong field.
3. Is there real depth?
Did the work uncover contradictions, competing truths, emotional tensions or non-obvious dynamics?
Or has it mainly organised familiar ideas particularly well?
If the long report does not materially deepen the summary, there may still be another layer to discover.
4. Has it interpreted, rather than just summarised?
Research should move beyond what people are saying to ask what those behaviours, expressions and tensions mean culturally, psychologically and commercially.
Observation is not yet interpretation.
5. Did the research improve the question?
Strong research should be capable of challenging the original brief.
If every conclusion fits comfortably inside the assumptions you started with, that may be reassuring.
It may also mean those assumptions were never meaningfully tested.
6. Is the evidence proportionate to the decision?
This may be the most important question of all.
How expensive would it be to be wrong?
How difficult would the decision be to reverse?
How much reputation, capital or organisational effort depends on the conclusion?
The higher the stakes, the stronger the evidence threshold should become.
Sometimes AI really is enough
This is important.
The answer should not always be “do more human research”.
Research has a tendency to sell more research.
We think the better standard is the minimum responsible evidence intervention.
Sometimes the AI-generated work is good enough.
Sometimes an existing body of human research already answers the question.
Sometimes one or two assumptions require validation.
Sometimes a handful of carefully designed ethnographic conversations would materially change the level of confidence.
Sometimes the problem needs netnography, observation or co-creation.
And sometimes the Human Reality Gap is substantial enough that the organisation genuinely needs to return to real people before moving forward.
The purpose is not to protect a methodology.
It is to protect the decision.
The AI Research Reality Check
This is why we are developing a new, lean, diagnostic at Trybes for people and organisations increasingly producing their own research with AI.
It is designed for consultants, strategists, marketing and innovation teams, founders, executives, agencies, newsletter writers and independent analysts — anyone whose decisions, recommendations or reputation increasingly depend on AI-assisted understanding of people.
The process is simple:
You give us the research, strategy, persona, analysis, report or article you are planning to use. And you tell us what you need it to support.
We examine the human assumptions, evidence base, methodological strength and decision risk underneath it.
The outcome is one of three broad conclusions:
🟢 AI is enough
The evidence is proportionate to the decision. There is no responsible reason to commission additional research. Keep moving!
🟠 Human validation would strengthen it
The overall direction is credible, but a small number of assumptions materially influence the conclusion and deserve testing. We identify exactly what those assumptions are and the lightest way to investigate them.
🔴 There is a Human Reality Gap
The recommendation depends on claims about people that are not sufficiently grounded in their lived reality. Before making the decision, the responsible next step is to return to real humans.
And even then, the objective is not automatically to design a large research programme.
It is to establish the smallest intervention capable of closing the gap.
Because AI is making answers extraordinarily cheap.
What becomes increasingly valuable is knowing which answers are safe to believe.
And, perhaps more importantly, which questions we still need real people to help us ask.
Before you act
If you are about to use AI-assisted research to shape a strategy, advise a client, brief a team, publish an analysis or support a significant decision, we can stress-test it before you move forward. Trybes’ AI Research Reality Check identifies where the evidence is strong, where assumptions are doing too much of the work, and where human research could materially change the conclusion. Send us one live piece of work and we’ll tell you how far you can safely trust it.
If you want to go deeper into these questions — and build some of the knowledge researchers use to distinguish useful evidence from false confidence — read The Boardroom Guide to Buying Better Research in the Age of AI or Customer Research & Customer Behaviour Research: How to Understand Customers and Their Decisions.