Column
Catching Insurance Fraud in the Call: A Multimodal NLP Pipeline for FNOL Detection
A hybrid pipeline combining ASR, speaker diarization, NER, LLM-based RAG, and speaker embeddings targets fraud at the First Notice of Loss stage. Synthetic data generation sidesteps the scarcity problem, while rule-based scoring flags narration reuse, structural inconsistencies, and repeated voiceprints.
Insurance fraud is most effectively intercepted at its earliest moment — the initial claim call.
Yet most fraud detection systems have focused on written documents and structured data. The spoken conversation between a claimant and an agent at the First Notice of Loss (FNOL) stage carries rich signals that have largely gone unanalyzed.
A June 2026 preprint by Akram, Htait, Sadka, Meisingseth, and Jaitly addresses this gap directly. The team built a multimodal hybrid NLP pipeline targeting FNOL-stage fraud and demonstrated that combining audio and text analysis outperforms text-only baselines.
Three things this article covers
- How the pipeline integrates ASR, speaker diarization, NER, LLM-based RAG, and speaker embeddings into a unified fraud detection architecture
- Why synthetic data generation is a practical solution to the real-data scarcity problem in insurance
- What claims operations and SIU teams can take from this research as a reference for building a claim prioritization system
① The pipeline architecture
The pipeline layers several distinct technologies.
Audio enters first. ASR (Automatic Speech Recognition) converts the call recording into text. Speaker diarization then segments that transcript by speaker — separating the agent’s turns from the claimant’s turns and establishing the dialogue structure.
On the text side, NER (Named Entity Recognition) extracts incident-relevant entities: dates, locations, vehicle or property identifiers, and named parties. Regular expression-based feature extraction adds structured signals on top.
LLM-based RAG (Retrieval-Augmented Generation) brings contextual reasoning. This module compares the claimant’s account against a knowledge base of prior cases and flags inconsistencies that simple pattern matching would miss.
Speaker embeddings form the audio-side counterpart to this semantic analysis. Each speaker’s voice is encoded as a numerical vector. This makes it possible to compare voice characteristics across calls and flag cases where the same voice appears under different identities.
The outputs from these layers feed into a rule-based scoring engine that produces a fraud suspicion score for each claim.
② Why synthetic data works here
One of the more practically significant aspects of this research is that it does not rely on real insurance fraud data.
In the insurance industry, fraud case records are among the most tightly controlled data assets. Privacy regulations, legal liability concerns, and competitive sensitivity make it nearly impossible for outside researchers to obtain them. This data scarcity is one of the central bottlenecks to fraud detection research.
The research team generates synthetic agent-claimant transcripts and two-speaker audio, constructing both fraudulent and legitimate call scenarios artificially. This bypasses the access problem entirely.
For claims operations teams considering a proof of concept, this matters. A PoC can be launched using synthetic data generated to mirror the characteristics of the target environment — without waiting for data access approvals or anonymization pipelines to be in place. The approach allows the technical case to be made before the organizational and legal groundwork is complete.
③ The three fraud signals the system targets
The pipeline is designed around three specific fraud indicators.
The first is narration reuse. Fraudulent claimants sometimes recycle nearly identical accounts of an incident across multiple claims or calls. Text similarity analysis and NER-based comparison detect this kind of templated storytelling.
The second is structural inconsistency. When the internal logic of a claimant’s account breaks down — timeline contradictions, location inconsistencies, relational incoherence among named parties — the NER-plus-RAG layer surfaces these discrepancies. The LLM provides the contextual judgment that rule-based pattern matching alone cannot.
The third is repeated voiceprint. Speaker embeddings allow the system to check whether the same voice has appeared across different claims. This targets scenarios where a single individual poses as multiple claimants, or where a fraud ring systematically generates claims from a small group of people.
These three signals are combined via rule-based scoring to produce a single fraud suspicion score per claim.
④ Use cases for claims operations and SIU teams
The natural application for this pipeline is a claim prioritization system embedded in FNOL call center operations.
Call recordings flow into the pipeline in real time or batch. The system returns a fraud suspicion score. Claims above a defined threshold are automatically flagged for SIU (Special Investigation Unit) review. Claims below the threshold proceed through standard assessment.
This changes how SIU capacity is allocated. SIU investigators are a finite resource. Manual triage based on examiner intuition and basic document review produces inconsistent coverage. Score-based triage concentrates investigator attention on the cases most likely to warrant it — and creates an auditable basis for escalation decisions.
For IT teams and claims operations managers evaluating this approach, the integration path is relatively direct if a call recording infrastructure is already in place. The pipeline extends that infrastructure with an analysis layer. It does not require replacing existing systems.
Compliance and AML functions in financial institutions should also note the relevance here. The structural fraud patterns that manifest in insurance FNOL calls — narration reuse, identity cycling, coordinated claim rings — share characteristics with financial fraud. The multimodal approach transfers.
For InsurTech teams, the synthetic data methodology provides a low-barrier entry point for client pilots. The ability to demonstrate the pipeline without accessing client data reduces the procurement and legal timeline considerably.
⑤ KPIs and implementation considerations
Three primary KPIs apply when evaluating this system in production.
SIU detection rate measures what fraction of AI-flagged claims are confirmed as fraud by investigation. This tells you whether the scoring signal corresponds to real fraud, not just surface-level anomalies.
False positive rate measures what fraction of legitimate claims are incorrectly escalated to SIU. High false positive rates increase investigator workload and erode trust in the system. Calibrating the threshold is a domain judgment call that requires input from experienced examiners.
Claims review lead time measures whether score-based triage shortens the end-to-end cycle from FNOL to resolution. This can be decomposed into time-to-SIU-escalation and time-to-final-decision for both escalated and non-escalated claims.
Several implementation considerations deserve attention. The pipeline was validated on synthetic data. Real-world performance on live call center audio — with background noise, accents, overlapping speech, and telephony artifacts — will need independent evaluation. ASR accuracy, in particular, is sensitive to acoustic conditions and language variety.
The rule-based scoring thresholds require domain calibration. Setting the right escalation cutoff involves a tradeoff between detection rate and false positive rate that is specific to the claims environment, fraud prevalence, and SIU capacity. This calibration is not a one-time step — it should be revisited as fraud patterns evolve.
Finally, any deployment that processes voice data must address privacy and consent requirements specific to the jurisdiction. Recording notice obligations, data retention policies, and the use of voice biometrics each carry regulatory dimensions that vary across markets.
The reframe worth taking away
Most fraud detection investment goes into document review, database cross-checks, and post-claim investigation. The FNOL call — where the fraud is being narrated in real time — has been an underutilized signal source.
The contribution of this research is to establish that multimodal analysis at the FNOL stage is technically feasible and outperforms text-only approaches. Synthetic data generation makes it accessible for research and early-stage PoC without the data access barriers that have historically limited this area.
The reframe for claims operations leadership is not “AI as a fraud detector” but “AI as a triage layer that makes human investigators more effective.” The pipeline does not replace SIU judgment. It concentrates that judgment where it is most needed.
In an environment where fraud sophistication is increasing and investigation resources are not, that concentration effect is where the value lies.
Reference
- Muhammad Shakeel Akram, Amal Htait, Abdul Hamid Sadka, Emma Meisingseth, Karishma Jaitly (2026). Dialogue to Detection: A Multimodal Hybrid NLP Pipeline for Insurance Fraud Detection. arXiv preprint. https://arxiv.org/abs/2606.28002
* This article was written in part with AI assistance and may contain inaccuracies.