30 citations · 92 across the 15 of their papers we have counts for
15 papers
Expresso-AI: Explainable Video-Based Deep Learning Models for Depression Diagnosis
Felipe Moreno, Sharifa Alghowinem, Hae Won Park +1
Given the widespread prevalence of depression and its consequential impact on individuals and society, it is crucial to obtain objective measures for early diagnosis and interventi…
EmpathicStories++: A Multimodal Dataset for Empathy towards Personal Experiences
Jocelyn Shen, Yubin Kim, Mohit Hulse +4
Modeling empathy is a complex endeavor that is rooted in interpersonal and experiential dimensions of human interaction, and remains an open problem within AI. Existing empathy dat…
HEART-felt Narratives: Tracing Empathy and Narrative Style in Personal Stories with LLMs
Jocelyn Shen, Joel Mire, Hae Won Park +2
Empathy serves as a cornerstone in enabling prosocial behaviors, and can be evoked through sharing of personal experiences in stories. While empathy is influenced by narrative cont…
Improving Dialogue Agents by Decomposing One Global Explicit Annotation with Local Implicit Multimodal Feedback
Dong Won Lee, Hae Won Park, Yoon Kim +2
We describe an approach for aligning an LLM-based dialogue agent based on global (i.e., dialogue-level) rewards, while also taking into account naturally-occurring multimodal signa…
MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making
Yubin Kim, Chanwoo Park, Hyewon Jeong +7
Foundation models are becoming valuable tools in medicine. Yet despite their promise, the best way to leverage Large Language Models (LLMs) in complex medical tasks remains an open…
Integrating Flow Theory and Adaptive Robot Roles: A Conceptual Model of Dynamic Robot Role Adaptation for the Enhanced Flow Experience in Long-term Multi-person Human-Robot Interactions
Huili Chen, Sharifa Alghowinem, Cynthia Breazeal +1
In this paper, we introduce a novel conceptual model for a robot's behavioral adaptation in its long-term interaction with humans, integrating dynamic robot role adaptation with pr…
Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Data
Yubin Kim, Xuhai Xu, Daniel McDuff +2
Large language models (LLMs) are capable of many natural language tasks, yet they are far from perfect. In health applications, grounding and interpreting domain-specific and non-l…
HIINT: Historical, Intra- and Inter- personal Dynamics Modeling with Cross-person Memory Transformer
Yubin Kim, Dong Won Lee, Paul Pu Liang +3
Accurately modeling affect dynamics, which refers to the changes and fluctuations in emotions and affective displays during human conversations, is crucial for understanding human…
Modeling Empathic Similarity in Personal Narratives
Jocelyn Shen, Maarten Sap, Pedro Colon-Hernandez +2
The most meaningful connections between people are often fostered through expression of shared vulnerability and emotional experiences in personal narratives. We introduce a new ta…
Multipar-T: Multiparty-Transformer for Capturing Contingent Behaviors in Group Conversations
Dong Won Lee, Yubin Kim, Rosalind Picard +2
As we move closer to real-world AI systems, AI agents must be able to deal with multiparty (group) conversations. Recognizing and interpreting multiparty behaviors is challenging,…
Joint Engagement Classification using Video Augmentation Techniques for Multi-person Human-robot Interaction
Yubin Kim, Huili Chen, Sharifa Alghowinem +2
Affect understanding capability is essential for social robots to autonomously interact with a group of users in an intuitive and reciprocal way. However, the challenge of multi-pe…
Explainable AI for Suicide Risk Assessment Using Eye Activities and Head Gestures
Siyu Liu, Catherine Lu, Sharifa Alghowinem +3
The prevalence of suicide has been on the rise since the 20th century, causing severe emotional damage to individuals, families, and communities alike. Despite the severity of this…
The Interaction Flow Editor: A New Human-Robot Interaction Rapid Prototyping Interface
Matthew Huggins, Anastasia K. Ostrowski, Andrew Rapo +3
Human-robot interaction can be regarded as a flow between users and robots. Designing good interaction flows takes a lot of effort and needs to be field tested. Unfortunately, the…
A Robotic Positive Psychology Coach to Improve College Students' Wellbeing
Sooyeon Jeong, Sharifa Alghowinem, Laura Aymerich-Franch +5
A significant number of college students suffer from mental health issues that impact their physical, social, and occupational outcomes. Various scalable technologies have been pro…
Dyadic Speech-based Affect Recognition using DAMI-P2C Parent-child Multimodal Interaction Dataset
Huili Chen, Yue Zhang, Felix Weninger +3
Automatic speech-based affect recognition of individuals in dyadic conversation is a challenging task, in part because of its heavy reliance on manual pre-processing. Traditional a…