Column
The Real Reasons Behind Disengagement Won't Show Up in Your Survey
Why did NPS drop? Why is engagement down? The honest answers rarely survive a multiple-choice survey. A 571-person experiment just proved that AI conversational interviewing can surface those hidden insights at scale — here's what that means for HR, market research, and VoC teams.
Hi, I’m Keito Inoshita from Affectosphere Group.
Conversations with HR and market research teams tend to circle back to the same frustration.
“NPS is down, but the survey doesn’t tell us why.”
“Engagement scores dropped, but we can’t figure out what employees are actually unhappy about.”
“We want to go deeper, but one-on-one interviews take too much time and money.”
There’s always been an unbridgeable gap between quantitative surveys and qualitative interviews.
Surveys scale but stay shallow. Interviews go deep but don’t scale. This either-or has been the structural limitation of customer and employee research for decades.
A 2026 arXiv paper (Alexander Wuttke, Max Melchior Lang, Christopher Klamm, Quirin Würschinger, Frauke Kreuter; arXiv:2606.20064) reports a 571-person experiment that breaks through this constraint. It demonstrates that AI conversational interviewing can function as a scalable qualitative research infrastructure.
Today I’ll break down what this means for HR, market research, and engagement management teams who want to act on it.
3 Points for Today
- Value: AI conversational interviewing collects the kind of insights surveys can’t surface — at scale.
- What the research found: What a 571-person experiment revealed about the gap between conversational interviews and standardized surveys.
- How to use it in practice: Concrete use cases for VoE and VoC collection.
① Why Surveys Miss the Real Story
First, the structural problem.
Quantitative surveys have a fundamental limitation: the moment you design the response options, you’ve defined the ceiling of what you can learn.
“How satisfied are you with your work environment? ① Very satisfied ② Somewhat satisfied ③ Somewhat dissatisfied ④ Very dissatisfied.”
That question cannot capture “I like my team, but I can’t see growth opportunities, so I’m thinking about leaving.” It also can’t detect “the issue isn’t compensation — it’s that psychological safety is low and I don’t feel heard.”
A human interviewer running a semi-structured interview can follow up: “When you say somewhat dissatisfied, can you walk me through a specific situation?” But doing that with hundreds or thousands of respondents isn’t realistic.
That’s exactly where AI conversational interviewing fits.
② What the 571-Person Experiment Revealed
The researchers designed an experiment around immigration policy opinions in Germany, with 571 participants and three interview formats: voice, chat, and hybrid (chat plus voice).
The most important finding: different mental models exist across subgroups — and standardized surveys miss them entirely.
A traditional survey tends to sort respondents along a single axis: “support vs. oppose” immigration policy. But when the researchers analyzed the conversational transcripts, they found that respondents with the same stated position had arrived at it through entirely different logic, emotional reasoning, and underlying values.
“Same answer, completely different reasoning” — that’s a finding that simply cannot come from a multiple-choice survey.
Across the three interview formats, all three produced richer data than standardized surveys. There were differences in response depth and participant comfort depending on the format, which is directly useful when deciding which mode to deploy in a real research context.
The paper also emphasizes that AI conversational interviewing functions as a complement to human interviewers, not a replacement. In situations requiring nuanced probing or emotional sensitivity, human interviewers remain necessary. That’s an honest and important framing for practitioners.
③ How to Use It — Three Practical Use Cases
Here’s where theory meets the real world.
Use Case 1: “Why” Layer for Annual Engagement Surveys
Most companies run engagement surveys once or twice a year.
But what’s happening in June — right after a major restructuring in March — stays invisible until the next survey cycle. That gap is where talent walks out the door without anyone understanding why.
AI conversational interviewing fills this gap with an always-on listening model.
Concrete implementation: Deploy monthly AI conversational interview sessions to teams whose engagement scores dropped. Start with “Have you noticed any changes in your work recently?” and follow with “Can you describe a specific situation where you felt that?” The transcripts are auto-aggregated and theme-classified.
KPI: Percentage of at-risk employees whose reasons for disengagement were identified before resignation (vs. baseline where the reason was unknown at exit).
Use Case 2: Root Cause Analysis When NPS Drops
NPS dropped five points vs. last quarter. The open-text survey comments say things like “slow response” and “hard to use” — but nothing specific enough to act on.
Run 15-minute AI conversational interview sessions with 100 Detractor respondents (NPS 7 and below). Start with “Can you walk me through the last time you used our service?” and probe with “When you say it felt hard to use, which step did that happen at?”
Auto-aggregate and classify the 100 conversation transcripts. The root cause that emerges might not be UX at all — it might be that first-contact resolution rates in customer support are low. That’s an actionable finding that survey free-text fields would never surface.
KPI: Time to root cause identification, NPS improvement rate in the following quarter.
Use Case 3: Behavior Change Support After 360-Degree Feedback
A manager receives 360 feedback saying “communication needs improvement.” But “communication” is too broad to act on.
Run AI conversational interview sessions with both the manager and team members. Ask: “Can you describe a recent situation where you felt communication broke down?” This surfaces specific moments and contexts.
By mapping the gap between how the manager perceives their communication and how team members experience it, you move from vague feedback to an actionable hypothesis: “The issue isn’t meeting frequency — it’s the timing and quality of feedback delivered in 1-on-1s.”
Designing “Always-On Listening” as Infrastructure
The bigger possibility this research points to is a paradigm shift in research design itself.
Traditional research is event-driven. Annual engagement surveys. Post-purchase questionnaires. Pre-launch user interviews. All point-in-time data collection.
When AI conversational interviewing is cheap and scalable, time-series data collection becomes possible. Sessions can be triggered automatically by specific events: pre-exit interview, post-NPS drop, post-product update. Instead of asking “how are things?” once a year, you listen continuously.
This is a shift from “an organization that surveys” to “an organization that listens.”
The implementation barrier is lower than it looks. As the paper argues, AI doesn’t need to fully replace human interviewers. Using AI as a triage layer — automatically routing cases that need deeper probing to human interviewers — lets you balance cost and depth at scale.
Scale, low cost, and deep insight. For teams that have been trying to achieve all three simultaneously, this research offers a concrete path forward.
That’s all for today!
References
- Alexander Wuttke, Max Melchior Lang, Christopher Klamm, Quirin Würschinger, Frauke Kreuter (2026). AI Conversational Interviewing: Scaling Up Semi-Structured and In-depth Interviews. arXiv preprint.
* This article was written in part with AI assistance and may contain inaccuracies.