Skip to content
Affectosphere Group Keito Inoshita Laboratory
JP

Research

Development Based on CAI

Development research that implements emotion-reading AI as applied systems and delivers them to society.

What we work on

Whether CAI is truly useful depends on whether it can function safely in psychologically delicate settings. In domains where the parties involved are highly vulnerable — family relationships, mental health, classrooms — implementation calls for the safety, empathy, and calibration of the design more than for the technical accuracy of the model. This domain takes as its object the methodology of implementing emotion-handling AI as applied systems and explores how to design psychologically safe, empathic, and practical feedback.

What we are finding

Implementations of CAI in psychologically delicate domains have yielded design knowledge that goes beyond technical accuracy. A design in which multiple LLM agents with specialist knowledge engage in role-play and multi-stage discussion is effective for producing empathic and practical feedback. At the same time, systems tend to exhibit excessive confidence in their own judgments, and calibration is shown to be indispensable for psychological-support applications.

Research notes

Research narrative

Working where it must not break

Whether CAI is truly useful depends on whether it can function safely in psychologically delicate settings. Implementations in domains where the parties involved are highly vulnerable — family relationships, mental health, classrooms, elder care — carry challenges of a different order from lab-based accuracy evaluation. Users stand in a position where they can be deeply harmed by AI judgments, and a mistaken judgment from the AI can damage real human relationships. This domain takes as its object the methodology of implementing emotion-handling AI as applied systems and explores how to design psychologically safe, empathic, and practical feedback.

The constraint of clinical safety

AI applications in psychological-support areas face several distinctive difficulties. First, 'clinical safety': there is a risk that erroneous advice from the AI worsens the user's mental state. Second, the necessity of an 'empathic response': what is technically correct is not necessarily emotionally appropriate. Telling a user who reports sadness that 'objective data shows things will improve' may be correct but not appropriate. Third, the limits of 'substituting for experts': AI should not replace licensed professionals but remain in a complementary position. Research in this domain carefully designs the range in which AI can safely contribute, taking these difficulties into account.

Multi-agent debate produces useful feedback, and too much confidence Agents in role, debating detect interpret challenge synthesise empathetic, usable and very sure of itself stated confidence actual reliability the gap to calibrate In psychologically delicate settings, closing that gap matters more than the last point of accuracy.
役割を持つエージェントが多段で議論すると、共感的で実用的な出力が得られる。ただしシステムは自分の判断に自信を持ちすぎる。心理支援では、この差を詰めることが精度より重い。

A framework for family communication

Role-Playing LLM-Based Multi-Agent Support Framework (2025) proposes a system that detects 'suppressed emotion' in a child and 'ideal-parent bias' in a parent from parent–child conversation and returns empathic feedback to each family member. 'Suppressed emotion' refers to feelings the child has toward the parent but cannot directly express, while 'ideal-parent bias' refers to the cognitive bias by which the parent evaluates themselves as an 'ideal parent' and thereby overlooks the child's actual experience. The system is built on 30 Japanese parent–child dialogue scenarios and integrates four processing stages: suppressed-emotion detection, attribute estimation, bias detection, and a five-agent debate.

Five roles, played apart

The 'five-agent debate' at the core of this system is a design in which five roles — psychologist, educator, parent role, child role, and moderator — are played by different LLMs, each analyzing and critiquing the conversation from its own standpoint. Having LLMs perform role-play allows the design to structurally surface points that a single perspective overlooks. The psychologist agent contributes psychodynamic insight into suppressed emotion, the educator agent contributes responses suited to developmental stage, the parent-role and child-role agents contribute first-person perspectives, and the moderator integrates them. By mechanically reproducing a discussion among specialists with diverse perspectives, the system produces empathic and practical feedback.

Judged by people, run by machine

Evaluation was conducted in two stages. In the first stage, human evaluation, empathy and practicality scored highly. In the second stage, simulation dialogues, signs of reduced suppression and improved mutual understanding were observed in parent–child dialogues after feedback. At the same time, the system exhibited a tendency toward excessive confidence in its own judgments — an important finding that calibration (agreement between confidence and correctness) is indispensable for psychological-support applications. This connects directly to uncertainty research in our lab's interpretability domain and shows the need for dialogue between applied development and basic research.

What accuracy cannot measure

Evaluation of CAI in psychological-support domains has facets that ordinary NLP-task evaluation cannot capture. Beyond technical accuracy (agreement with ground-truth labels), a multifaceted set of indicators is required: (i) empathy (whether the user feels emotionally received), (ii) practicality (whether it leads to concrete behavioral change), (iii) safety (whether the response avoids harming the user), (iv) explainability (whether the system can state why it gives a given piece of advice), and (v) calibration (whether confidence expression is appropriate). This work shows one example of such a multifaceted evaluation framework and makes a pioneering contribution to how CAI should be evaluated in delicate domains.

A template for delicate ground

This work offers a methodological template for deploying CAI to delicate interpersonal domains. Designing the safety and explainability of the system's behavior together with the technology itself is becoming a central concern of implementation research. Going forward we plan to extend beyond family relationships to education, elder care, workplace mental health, and other applied areas. We will advance the social deployment of CAI in concert with normative debate in the ethics-and-philosophy domain, uncertainty research in the interpretability domain, and the design of relationships in the human–AI interaction domain.

Work in this domain

Publications in this domain

4 entries / Interaction

See all publications →

Other research domains