Affective AI and Art

Art is the densest expression of emotion and at the same time the most open site of interpretation. Painting, music, literature, and the performing arts all transmit the creator's emotional experience to the receiver through a medium and evoke emotional experience on the receiver's side. Since antiquity, art has functioned as a cultural device for understanding and sharing human emotion. The intersection of affective AI and art is a domain that adds a new technical layer to this cultural tradition. This domain explores how AI that handles emotion can function in the territory of art and how it can extend or constrain human creativity and sensibility.

With the rise of generative AI, the positions AI can take with respect to art can be organized into three large categories. The first is 'making,' in which AI directly generates artworks. Generative AI such as Stable Diffusion, DALL-E, Sora, and Suno produces images, video, and music from textual descriptions and has dramatically lowered the threshold of creation. The second is 'reading,' in which AI interprets existing artworks and extracts emotion, meaning, and context. Emotion analysis of paintings, analysis of affective structure in music, and extraction of emotion curves from literary texts fall here. The third is 'inspiring,' in which AI takes on the supportive role of helping and prompting human creation — idea generation, style transfer, critical feedback. Our lab pursues research that moves among these three positions.

Numerous serious ethical issues surround AI art. First, copyright of training data: generative AI uses large numbers of existing works as training data, yet their creators have typically not consented to such use. Second, ownership of copyright in AI-generated works: which entity owns the copyright in works generated by AI is a question that is still being worked out across jurisdictions. Third, the problem of substituting for or suppressing human creativity: if cheap AI generation becomes commonplace, the economic foundation of human artists can be destroyed. Fourth, cultural diversity: generative AI reflects the bias of its training data and tends to generate output biased toward particular cultural styles, which can reduce cultural diversity as a result. This domain proceeds in parallel with these issues and conducts art research that draws on the distinctive features of affective AI.

The distinctive contribution affective AI can offer to art is 'fine-grained interpretation of emotion' and 'adjustment of expression according to emotion.' Examples include museum interfaces that change how a work is presented according to the user's current emotional state, tools that visualize the emotion curves of literary works to support criticism, and systems that analyze the affective structure of music to support composition — there are many possible combinations of affective AI and art. The techniques accumulated in our lab's domains of emotion recognition, data augmentation, and interpretability can all serve as foundations for application in art.

At present this domain is at a stage of launching projects, and the number of published papers is limited. Nonetheless, the intersection of affective AI and art is positioned as an important axis for the lab's future development. In parallel with exploring technical possibilities, we will develop research that consciously connects to humanistic practices — creation, criticism, education, curation. By moving deliberately among the three positions of 'making,' 'reading,' and 'inspiring,' we aim to present new forms of art in which human sensibility and AI capability complement each other.