Do We Still Need to Ask Humans? The Theoretical Case for LLMs as Statistical Estimators

Hello. This is Keito Inoshita from Affectosphere Group.

Every market researcher knows the problem. Incentive budgets, low response rates, cleanup time, bias correction — getting accurate customer insight is slow and expensive. The question that keeps coming up is: could we use LLMs instead?

Asking an LLM to respond “as a 30-something female consumer” has been tried. The outputs feel plausible. But “feels plausible” is not a statistical guarantee, and without one, it’s hard to build a product or research process around LLM-generated responses.

A June 2026 study (Yang, arXiv:2606.30372) addresses that gap directly. Using the framework of Le Cam deficiency analysis, it asks whether pre-trained LLMs can function as risk-equivalent estimators of conditional expectations in human-response data — and demonstrates that, under appropriate calibration, they can.

This moves the conversation from “it seems to work” to “here is why it works.”

---

Three things this article covers

• A well-calibrated LLM can achieve Bayes-optimal statistical performance under squared loss — proven via Le Cam deficiency analysis
• This provides theoretical grounding for using LLMs as substitutes for human-subject experiments in social and behavioral science
• Practical survey design can use LLMs to reduce cost while preserving statistical rigor — if the scope is defined carefully

---

① What “Statistical Estimator” Actually Means

An estimator is a function that uses observed data to infer a true value. When you want to know how 30-something male consumers feel about a new product, you sample — you can’t ask everyone. Statistical theory provides the framework for evaluating how good that estimate is.

This study shows that a pre-trained LLM functions as a risk-equivalent estimator of conditional expectations under squared loss. Le Cam deficiency analysis measures how much statistical information is lost when you substitute one experimental setup for another. The paper uses this framework to quantify the gap between a human-subject experiment and an LLM estimation experiment.

The finding is that a properly calibrated LLM can close that gap to the point of Bayes-optimal performance. This is a formal mathematical result, not a qualitative claim.

---

② Le Cam Deficiency: The Framework

Le Cam deficiency analysis compares two statistical experiments in terms of information content. Frame “run a human survey” and “query an LLM” as two experiments, then measure how much statistical information the second loses relative to the first.

When the deficiency is small — when the LLM experiment retains nearly as much information as the human experiment — you have theoretical grounds for substitution. The study proves that well-calibrated LLMs can reach this threshold.

Calibration here means adjusting the LLM’s output distribution to match the empirical distribution of human responses. This step is what converts the theoretical result into practical validity. Without calibration on real human data, the guarantee does not apply.

---

③ What Is and Is Not Substitutable

Overstating this finding would be a mistake. The theory covers statistical estimation of collective response patterns — not individual human experience, not cultural contexts outside the training distribution, not groups the model has not meaningfully seen.

The claim is: for populations well-represented in pre-training data, LLMs can estimate the typical distributional response to a stimulus with Bayes-optimal accuracy, when properly calibrated.

That scope is significant but bounded. Unique individual experiences, emerging demographic behaviors, and highly localized contexts remain outside the theoretical guarantee. Knowing this boundary is what makes the tool usable responsibly.

---

④ Implications for Social and Behavioral Science

The study highlights the potential to conduct large-scale human-subject research at dramatically lower cost in social and behavioral science.

The replication crisis in psychology is partly a sample-size crisis. Collecting adequate samples is expensive, and under-powered studies make claims that don’t hold up. If LLM-generated data can substitute for a portion of human-subject collection with theoretical backing, a two-stage design becomes possible: use LLMs for broad exploratory sampling, then use human subjects to confirm the hypotheses that matter most.

This expands the range of hypotheses researchers can test within a fixed budget — a structural improvement in research design rather than a corner-cut.

---

⑤ A Practical Design for Customer and Employee Surveys

Consider a marketing team that runs large biannual customer satisfaction surveys. Here is a concrete way to apply this research.

First, decompose survey objectives. “How does the 35-44 demographic evaluate this feature?” is a distributional question — suited for LLM estimation. “Why did a specific cohort churn?” is a causal explanation question — suited for human interviews.

For the LLM estimation phase, calibrate the model on historical survey data. Match LLM output distributions to observed human response distributions from past waves. This is the step that satisfies the theoretical conditions. Then run large-scale simulations of new initiatives.

For the human survey phase, target the questions where LLM uncertainty is highest or where historical calibration data is sparse. Reduce the questionnaire scope and incentive spend to cover only confirmatory questions.

KPI suggestion: track insights per dollar of survey spend before and after LLM integration. Count “statistically significant findings” as the unit — and see whether LLM-augmented design yields more per dollar invested.

---

Conclusion

Using LLMs as survey proxies is not a new idea. But until now it has been an intuition, not a theory. This study provides the statistical framework that was missing: a formal argument, grounded in Le Cam deficiency analysis, for why a well-calibrated LLM can substitute for human-response data in a principled way.

The shift from “this seems to work” to “this is why it works” matters. It opens the door to building survey products and research designs that treat LLM estimation as a first-class statistical tool — not a shortcut, but an alternative with known scope and theoretical backing.

---

Reference

1. Yang, Haobo. (2026). Using Large Language Models as Low-Cost Statistical Estimators for Human-Response Data. arXiv preprint (2026).

* This article was written in part with AI assistance and may contain inaccuracies.