The future of customer research

One of the product leads who supported our early work wanted to see his product managers in the office no more than one day a week. He wanted them out spending time with customers.

When we started PoDojo in 2011, our focus was customer-centred agile product development. One of our biggest surprises was how few of the product managers attending our Product Dojos were conducting customer interviews themselves. For many, going out and speaking to customers was unfamiliar territory.

Helping them do that became a central part of our work. Discovery interviews, small experiments and usability tests became regular ingredients of our Dojos. We were enthusiastic about Lean Startup and Customer Development, and about helping teams learn what customers needed while they were still figuring out what to build.

We weren't researchers. My own understanding of research came from product management. I had read Steve Portigal, and once shared a fish platter with him at the Productized conference in Lisbon. But I had a fairly naive idea of how professional research worked inside organisations and what researchers considered sufficient evidence for a claim.

Two experiences that changed my view of research

In one project, we were working with a medical technology startup. Clinical trials were part of its work, so the team needed scientific expertise and evidence that could meet the standards of those trials. When we proposed our lean experiments, the researcher was very surprised. He didn't really take us seriously, and his comments on the statistical validity of our results left us feeling painfully naive.

We had been using the language of hypotheses, experiments and validation, drawing on Lean Startup and David Bland's Testing Business Ideas. But we were mixing two domains: clinical research needed scientifically defensible findings, while product discovery needed to help us find customer value and decide what to try next. The startup needed both, but we struggled to distinguish which questions we were answering and what kind of evidence each required. I realised that words like "validated" made our findings sound more certain than they were. I felt embarrassed, as if we had been showing off with scientific language without really knowing what we were talking about.

But I still believed that provisional findings could help us make better product decisions, provided we stayed honest about how little we knew and kept those decisions within the limits of our evidence.

In another project, the tension was about time and scope. The researcher wanted to investigate the field broadly and thoroughly. The rest of us, working across product, engineering and design, wanted to do guerrilla research and run small tests with customers. We were already short of time.

It became a serious conflict. Looking back, I can see something legitimate in both positions. We needed enough understanding to make sensible decisions, and we needed to make those decisions while the project could still act on them. At the time, we didn't manage to bring those needs together.

I still think about that project and what we could have done differently. How could the things we learn quickly help us decide where to investigate more deeply, and how could deeper research make our next experiments better?

My background is in product management and computational linguistics, and I'm still learning through working with AI and alongside professional researchers. These are a few thoughts from that experience, which I hope are useful to others exploring the same questions.

Making continuous discovery easier

Our current work with AI has brought me back to that question. At PoDojo, we are working with AI-moderated interviews, automated testing and ways for teams to ask questions of their existing research. Along with automated recruiting, these capabilities can reduce the effort needed to sustain the continuous discovery habits described by Teresa Torres.

I would not assume that this means teams will spend more time with customers or learn more from them. Which interviews we automate also depends on the empathy and understanding of our customers’ world we want to develop ourselves. Direct contact is the best way to build that understanding. For exploratory research, spending time with customers and interviewing them ourselves belongs in our toolkit. I would not outsource that learning to AI, any more than I would to a marketing or market research agency. Teams still need the time and willingness to engage with customers and act on what they learn.

Let's assume a team wants to add an idea to the roadmap. An agent working with a shared knowledge base could help them assess it against existing research and relevant product context before they commit to it. They could see what evidence supports the idea, where they are still making assumptions and which questions need further research. Depending on what they find, the next step might be to move forward, run a small test or investigate more deeply.

Observations from these conversations and tests need to become part of the same body of knowledge as deeper research, with their different limits still visible. We should be able to trace findings back to their sources and recognise when they no longer apply. Talking to the agent could bring up new questions, uncertainties or contradictions that become starting points for further research.

This is the connection I hope agentic systems can help us build. For this knowledge to compound, new evidence has to help us question, correct and deepen what we already know. Quick tests would have a better starting point, and deeper research would stay connected to the decisions teams are trying to make.

What researchers bring to the system

That is where I see a growing responsibility for professional researchers. Their deep domain knowledge and methodological expertise shape how the system works, including when it stops offering an answer and recommends further investigation. We recently put together a role profile for a Wiki Engineer. Describing the work of building and maintaining a research knowledge base helped us see how much it depends on research expertise.

For me, evals are the heart of this work. Researchers define what a well-supported answer looks like, what evidence it needs and when the system needs to acknowledge that it doesn't know enough. They turn those expectations into test cases based on the questions people actually ask.

These evals develop as we learn. If an insights wiki develops a strong bias, we investigate what is happening and add checks that make the problem visible. We then revise the synthesis, the selection of sources or the agent's instructions, and test whether those changes address the problem. This is an ongoing part of maintaining the system's quality. Alongside these evolving evals, we keep a stable set of questions to check whether answers improve as the knowledge grows.

Skills come next. They turn methodological knowledge into reusable instructions that guide how agents work with colleagues across the organisation. Researchers use them to help colleagues clarify the question, add missing product context and decide whether the available evidence can answer it, or whether a different research method is needed. As we find weaknesses, we revise the skills and check the results through our evals.

The third part is the social side of research. In shared Slack threads, product managers, researchers and agents work through questions together. The agent brings relevant evidence; people add context, ask questions and compare interpretations. Researchers help the group judge what the evidence supports and what to investigate next. These conversations develop a shared understanding of customers as decisions take shape, and show researchers where colleagues struggle to interpret the available knowledge. They also show which questions keep coming up.

Researchers keep doing research themselves. Their investigations deepen their understanding of the domain, reveal things the system misses and help them question answers that sound convincing. What they learn informs new studies and feeds back into the evals, skills and corrections that keep the shared knowledge useful.

That is what I mean by gardening: investigating the world the knowledge describes, bringing new observations into the system, correcting its knowledge and methods, and evaluating those changes. Before an observation enters the shared knowledge base, we check that it represents the source faithfully and keeps the context and limits of the evidence. Engineering builds and operates the technical infrastructure in collaboration with researchers, who take responsibility for its methodological quality and develop the criteria by which we judge it. For me, this is part of research's strategic role in helping organisations understand their customers and make informed decisions.

Learning by building

With coding agents such as Codex or Claude Code, everyone on a research team can quickly build small systems of their own. I think it is essential that everyone experiments with this themselves. Research is knowledge work, and working directly with agents, wikis and toolkits gives us a practical understanding of how to organise that work in a more agentic way. For me, the starting point is playful: building something, changing it and seeing what happens.

That does not mean everyone on the team takes responsibility for building and maintaining the shared team system. There is value in everyone gaining experience through their own experiments, while the team agrees on who develops and looks after the system they use together.

The second step is a pilot project. Researchers work with AI Engineers to develop a system around real research questions and product decisions. This is where the evals, skills and shared conversations described above become part of the team's daily work. The aim is to build something useful for the research team and other stakeholders across the organisation, helping them use what they know about customers to make better product decisions. We also test whether the answers actually help teams make better decisions.

When we started PoDojo in 2011, we wanted product managers to spend more time with customers. Today, we can reduce much of the work needed to make that a regular part of product development. Whether teams use the opportunity still depends on how they choose to work.

Those uncomfortable conversations about our Lean experiments taught me to look more carefully at the claims we made. That is why I want researchers involved in building the systems we use to inform product decisions. Research expertise gives us a basis for questioning an answer, and learning to express that expertise through evals and skills makes it available to others as they work.