By Brett Watkins, CEO, L&E Research

Grab your coffee (or your beverage of choice, no judgment here), because I want to talk about something that’s been on my mind as I watch our industry sprint headlong into the age of AI, synthetic data, and remote-everything research.
In-person qualitative research is still the most powerful tool brands have for understanding their customers. And I worry we’re talking ourselves out of using it.
Let me explain why that matters, and why a book recommended to me by a client in one of the largest brand companies in the world makes the case better than I ever could.

The Mystery No Dashboard Can Solve

David Scott Duncan’s The Secret Lives of Customers opens with a deceptively simple problem: a fictional café chain called Tazza is losing customers, and nobody can figure out why. They have the data. They have the metrics. They have the dashboards. And yet the answer eludes them, right up until someone does something radical.

They actually go talk to their customer.

Duncan’s central insight, rooted in the “jobs to be done” framework pioneered by the late Clayton Christensen at Harvard Business School, is this: customers don’t buy products. They “hire” products, services, and brands to do a specific job in their lives. And when that job goes undone, or when a competitor does it better, customers fire you. Quietly. Without a survey response, a complaint email, or a single data point to warn you it’s coming.

That’s the mystery. And here’s the uncomfortable truth for insights leaders: no amount of behavioral data, no synthetic persona, and no AI-generated summary can fully crack it. Not yet. Maybe not ever.

Because the job a customer is hiring your product to do isn’t always the job you think it is.

What “Jobs to Be Done” Really Demands

The “jobs to be done” framework is brilliant in its simplicity, but it’s deceptively hard to execute well. The reason? You cannot identify the real job — the functional need, the emotional driver, the social context — from a survey alone.

Think about it this way. If Tazza had sent a questionnaire asking “Why did you stop coming in?”, they would have gotten answers. Probably logical, coherent, reasonable answers. “Prices went up.” “Location wasn’t convenient.” “I found somewhere closer.”

But what they would have missed is the real story: the feeling, the context, the moment. The fact that Tazza used to feel like a neighborhood living room, and now it feels like an airport terminal. That’s not a multiple-choice answer. That’s a conversation.

The jobs-to-be-done framework demands what Duncan calls “market detective work”: observation, curiosity, and the willingness to follow a thread wherever it leads. And for that, you need to be in the room.

The Case for the Room

I’ve been in this industry for over 30 years. I’ve watched research evolve from phone surveys to online panels to AI-assisted synthesis. Every evolution has brought real value. I’m not here to relitigate any of that.

But every time we move further from the physical room, we give something up. And I think we’ve started to forget what that something is.

When a consumer sits across from a moderator in a well-designed focus group facility, things happen that cannot be replicated on a Zoom call, and certainly cannot be inferred from passive behavioral data. A participant holds a product, and her grip tells you something. Another person starts to answer a question and then hesitates, and a skilled moderator follows that hesitation into the most important insight of the entire session. And don’t get me started on sensitive subjects, where I have watched grown men cry because of a solution they wish existed as they provided care for an ailing parent or child.

No algorithm catches those moments. No transcript captures what they mean. Only a human in the room — curious, present, and trained — could make that call.

These are the moments that change product roadmaps. These are the moments that save brands from launching the wrong thing to the wrong people for the wrong reasons.

The AI Argument Is Real. It’s Just Incomplete.

I want to be fair here. The argument for AI-assisted research, remote qual, and even synthetic data is not wrong; it’s just incomplete.

AI is extraordinary at synthesis. It can process hundreds of open-ended responses in a fraction of the time a human team could. It can identify patterns across datasets that would take weeks to surface manually.

Remote qual has expanded access in ways that matter: geographically, logistically, economically. We use it. We believe in it. And we’ve built technology to do it well. But Synthesis and access are not the same things as insight.

Synthetic data can model what people may do based on what they’ve done before. But it can’t predict what they will do. No one yet has cornered that outcome, but great qualitative research, conducted in real time, with real humans, in the real world, is still the closest thing I have seen to achieving it.

Duncan’s Tazza eventually figures this out. The breakthrough doesn’t come from better data analysis: it comes from going out and listening. From being present. From having actual conversations and following the threads that no algorithm thought to pull.

What This Means for Insights Leaders

If you’re leading an insights function, I’d ask you one question: when was the last time you or your team actually sat behind the glass?

Not reviewed a transcript. Not read an AI summary. Sat. Behind. The glass.

Because here’s what I know after four decades of watching the best researchers in the business work: the magic doesn’t happen in the report. It happens in the room. It happens when a brand-side researcher watches a real customer struggle to open their packaging and suddenly, viscerally, understands the problem 27 pages of quant data failed to communicate.

That moment of human-to-human understanding is the foundation of every great insight. And it’s available to you. It’s one conversation away.

Customers have secret lives. David Scott Duncan is right about that. They have needs they can’t fully articulate, contexts they’ve never been asked about, and jobs they need done that your brand may not even know it’s competing to fill. The only way to uncover those secrets is to get close. To be present. To create the conditions for real conversation that no survey or synthetic persona can come close to matching.

If you aren’t sure, ask yourself when the real insights in your own life transpired with the people who matter most to you. I’m told the younger generations love their phones, but I’ve watched my nieces’ and nephews’ faces light up when I leaned in, when their grandparents leaned in. The screen didn’t do that. Presence did. Humans are still most revealing when engaged in person. We are a social species. We don’t make decisions in a vacuum. Qualitative research is the psychological tissue that connects the dots on sociological behavior, and the best of it still happens in person, in the room, face to face.

That’s why our motto is: we bring brands and people together to talk. That’s where the magic happens. That’s where smarter decisions get made.

The room still matters. More than ever.

Brett Watkins is the CEO of L&E Research, a qualitative research firm that has connected brands with their customers since 1984. L&E operates research facilities across the United States and maintains a proprietary panel of 1.6 million ID-validated US consumers, patients, and medical professionals.