3 citations · 4 across the 9 of their papers we have counts for
9 papers
A Modern System Recipe for Situated Embodied Human-Robot Conversation with Real-Time Multimodal LLMs and Tool-Calling
Dong Won Lee, Sarah Gillet, Louis-Philippe Morency +2
Situated embodied conversation requires robots to interleave real-time dialogue with active perception: deciding what to look at, when to look, and what to say under tight latency…
Leveraging Large Language Models to Identify Conversation Threads in Collaborative Learning
Prerna Ravi, Dong Won Lee, Beatriz Flamia +5
Understanding how ideas develop and flow in small-group conversations is critical for analyzing collaborative learning. A key structural feature of these interactions is threading,…
Social Human Robot Embodied Conversation (SHREC) Dataset: Benchmarking Foundational Models' Social Reasoning
Dong Won Lee, Yubin Kim, Denison Guvenoz +5
Our work focuses on the social reasoning capabilities of foundation models for real-world human-robot interactions. We introduce the Social Human Robot Embodied Conversation (SHREC…
Does "Reasoning" with Large Language Models Improve Recognizing, Generating, and Reframing Unhelpful Thoughts?
Yilin Qi, Dong Won Lee, Cynthia Breazeal +1
Cognitive Reframing, a core element of Cognitive Behavioral Therapy (CBT), helps individuals reinterpret negative experiences by finding positive meaning. Recent advances in Large…
Improving Dialogue Agents by Decomposing One Global Explicit Annotation with Local Implicit Multimodal Feedback
Dong Won Lee, Hae Won Park, Yoon Kim +2
We describe an approach for aligning an LLM-based dialogue agent based on global (i.e., dialogue-level) rewards, while also taking into account naturally-occurring multimodal signa…
HIINT: Historical, Intra- and Inter- personal Dynamics Modeling with Cross-person Memory Transformer
Yubin Kim, Dong Won Lee, Paul Pu Liang +3
Accurately modeling affect dynamics, which refers to the changes and fluctuations in emotions and affective displays during human conversations, is crucial for understanding human…
Multipar-T: Multiparty-Transformer for Capturing Contingent Behaviors in Group Conversations
Dong Won Lee, Yubin Kim, Rosalind Picard +2
As we move closer to real-world AI systems, AI agents must be able to deal with multiparty (group) conversations. Recognizing and interpreting multiparty behaviors is challenging,…
Multimodal Lecture Presentations Dataset: Understanding Multimodality in Educational Slides
Dong Won Lee, Chaitanya Ahuja, Paul Pu Liang +2
Lecture slide presentations, a sequence of pages that contain text and figures accompanied by speech, are constructed and presented carefully in order to optimally transfer knowled…
Style Transfer for Co-Speech Gesture Animation: A Multi-Speaker Conditional-Mixture Approach
Chaitanya Ahuja, Dong Won Lee, Yukiko I. Nakano +1
How can we teach robots or virtual assistants to gesture naturally? Can we go further and adapt the gesturing style to follow a specific speaker? Gestures that are naturally timed…