Overview
Research introduces the concept of relation-aware emotional support conversation, defining it as a novel task aimed at assessing the capacity of large language models (LLMs) to discern and leverage evolving interpersonal relationship dynamics for the provision of more effective emotional support. This initiative addresses a gap in existing emotional support conversation systems, which predominantly focus on one-on-one seeker-supporter interactions and individual emotional states, while largely neglecting the complexities of interpersonal relations within multi-party scenarios.
Research Context
Traditional emotional support conversation systems are characterized by their primary focus on individual emotional states and dyadic interactions between a seeker and a single supporter. This established paradigm overlooks the intricacies inherent in multi-party emotional support contexts, specifically the dynamics of interpersonal relationships that evolve during these interactions. The present work seeks to expand the scope of emotional support systems to encompass these underexplored relational aspects, proposing a shift towards systems that can actively capture and utilize relational dynamics.
Approach
To evaluate relation-aware emotional support capabilities in LLMs, the researchers constructed a new benchmark named RESCUE (Relation-aware Emotional Support Conversation Understanding and Evaluation Benchmark). This benchmark was derived from authentic couple and family interview conversations. The dataset comprises 191 samples, featuring 7,079 annotated turns, and totaling 1,064.8 minutes of video content. RESCUE is built upon extensive annotations detailing socio-emotional and support-related dynamics present in the conversational data.
The RESCUE benchmark defines six distinct tasks. These tasks are designed to evaluate two core capabilities essential for relation-aware emotional support:
- Relational Understanding: This capability assesses the model's ability to comprehend the underlying relationship dynamics.
- Relation-Sensitive Support: This capability evaluates the model's capacity to provide support that is appropriately tailored to the understood relational context.
The experimental phase involved testing ten different LLMs against these six tasks to ascertain their performance in relation-aware emotional support conversation.
Findings
Experiments conducted with ten different LLMs on the RESCUE benchmark revealed differential performance across tasks. Current models demonstrated relatively proficient performance on tasks that primarily relied on local emotional or intervention cues. However, a notable observation was the models' struggle with tasks identified as relation-intensive. These challenging tasks included:
- Relation pattern prediction
- Viewpoint prediction
- Support strategy prediction
These findings collectively suggest limitations in the current generation of LLMs regarding their capacity to model complex interpersonal relations and consequently make support decisions that are sensitive to these relations.
Why This Matters
The identified limitations of current LLMs in modeling interpersonal relations and making relation-sensitive support decisions indicate an area for future development in emotional support conversation systems. Addressing these limitations could lead to the creation of more sophisticated systems capable of offering support tailored to the nuanced dynamics of multi-party interactions.