Development Based on Affective AI

Whether affective AI is truly useful depends on whether it can function safely in psychologically delicate settings. Implementations in domains where the parties involved are highly vulnerable — family relationships, mental health, classrooms, elder care — carry challenges of a different order from lab-based accuracy evaluation. Users stand in a position where they can be deeply harmed by AI judgments, and a mistaken judgment from the AI can damage real human relationships. This domain takes as its object the methodology of implementing emotion-handling AI as applied systems and explores how to design psychologically safe, empathic, and practical feedback.

AI applications in psychological-support areas face several distinctive difficulties. First, 'clinical safety': there is a risk that erroneous advice from the AI worsens the user's mental state. Second, the necessity of an 'empathic response': what is technically correct is not necessarily emotionally appropriate. Telling a user who reports sadness that 'objective data shows things will improve' may be correct but not appropriate. Third, the limits of 'substituting for experts': AI should not replace licensed professionals but remain in a complementary position. Research in this domain carefully designs the range in which AI can safely contribute, taking these difficulties into account.

Role-Playing LLM-Based Multi-Agent Support Framework (2025) proposes a system that detects 'suppressed emotion' in a child and 'ideal-parent bias' in a parent from parent–child conversation and returns empathic feedback to each family member. 'Suppressed emotion' refers to feelings the child has toward the parent but cannot directly express, while 'ideal-parent bias' refers to the cognitive bias by which the parent evaluates themselves as an 'ideal parent' and thereby overlooks the child's actual experience. The system is built on 30 Japanese parent–child dialogue scenarios and integrates four processing stages: suppressed-emotion detection, attribute estimation, bias detection, and a five-agent debate.

The 'five-agent debate' at the core of this system is a design in which five roles — psychologist, educator, parent role, child role, and moderator — are played by different LLMs, each analyzing and critiquing the conversation from its own standpoint. Having LLMs perform role-play allows the design to structurally surface points that a single perspective overlooks. The psychologist agent contributes psychodynamic insight into suppressed emotion, the educator agent contributes responses suited to developmental stage, the parent-role and child-role agents contribute first-person perspectives, and the moderator integrates them. By mechanically reproducing a discussion among specialists with diverse perspectives, the system produces empathic and practical feedback.

Evaluation was conducted in two stages. In the first stage, human evaluation, empathy and practicality scored highly. In the second stage, simulation dialogues, signs of reduced suppression and improved mutual understanding were observed in parent–child dialogues after feedback. At the same time, the system exhibited a tendency toward excessive confidence in its own judgments — an important finding that calibration (agreement between confidence and correctness) is indispensable for psychological-support applications. This connects directly to uncertainty research in our lab's interpretability domain and shows the need for dialogue between applied development and basic research.

Evaluation of affective AI in psychological-support domains has facets that ordinary NLP-task evaluation cannot capture. Beyond technical accuracy (agreement with ground-truth labels), a multifaceted set of indicators is required: (i) empathy (whether the user feels emotionally received), (ii) practicality (whether it leads to concrete behavioral change), (iii) safety (whether the response avoids harming the user), (iv) explainability (whether the system can state why it gives a given piece of advice), and (v) calibration (whether confidence expression is appropriate). This work shows one example of such a multifaceted evaluation framework and makes a pioneering contribution to how affective AI should be evaluated in delicate domains.

This work offers a methodological template for deploying affective AI to delicate interpersonal domains. Designing the safety and explainability of the system's behavior together with the technology itself is becoming a central concern of implementation research. Going forward we plan to extend beyond family relationships to education, elder care, workplace mental health, and other applied areas. We will advance the social deployment of affective AI in concert with normative debate in the ethics-and-philosophy domain, uncertainty research in the interpretability domain, and the design of relationships in the human–AI interaction domain.