10 citations · 24 across the 34 of their papers we have counts for
25 papers · 1 filter
TEIDAN: A Multilingual Multiparty Dialogue Corpus
Taiga Mori, Koji Inoue, Mikey Elmers +2
Multi-party interaction is a central setting for human communication and a necessary target for human-agent interaction systems that must participate in group conversation. Yet ava…
CultureConverse: A Multilingual Multi-turn Simulation Harness for Culturally Grounded Assistance in East and Southeast Asia
Bryan Chen Zhengyu Tan, Weihua Zheng, Thong T. Doan +30
Current cultural evaluations for large language models (LLMs) often reduce culture to single-turn factual recall via MCQs, failing to capture a common use case: users seeking pract…
MemUse: Moving Memory Evaluation from Direct QA to Natural Integration in Long-Term Human-AI Conversation
Ryuichi Sumida, Koji Inoue, Tatsuya Kawahara
Memory systems for conversational LLMs are conventionally evaluated by direct, fact-seeking questions about prior dialogue (Direct QA): can the model recall fact X from a prior con…
On the Structure of Address in Multi-Party Dialogue: From Discrete Labels to Continuous Levels
Taiga Mori, Koji Inoue, Divesh Lala +1
In multi-party dialogues between a dialogue system and multiple users, identifying to whom an utterance is addressed is a key challenge. Prior work has typically treated addressee…
I Understand How You Feel: Enhancing Deeper Emotional Support Through Multilingual Emotional Validation in Dialogue System
Zi Haur Pang, Yahui Fu, Koji Inoue +1
Emotional validation - explicitly acknowledging that a user's feelings make sense - has proven therapeutic value but has received little computational attention. Emotional validati…
Toward Signing Activity Projection in Sign Language Interaction
Takao Obi, Wang Yusong, Koji Inoue +1
Social robots must interact robustly not only with users assumed by speech-centered systems but also with diverse users whose communication relies on different modalities, e.g., si…
Emotion Transcription in Conversation: A Benchmark for Capturing Subtle and Complex Emotional States through Natural Language
Yoshiki Tanaka, Ryuichi Uehara, Koji Inoue +1
Emotion Recognition in Conversation (ERC) is critical for enabling natural human-machine interactions. However, existing methods predominantly employ categorical or dimensional emo…
Multilingual and Continuous Backchannel Prediction: A Cross-lingual Study
Koji Inoue, Mikey Elmers, Yahui Fu +5
We present a multilingual, continuous backchannel prediction model for Japanese, English, and Chinese, and use it to investigate cross-linguistic timing behavior. The model is Tran…
MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation
Weihua Zheng, Zhengyuan Liu, Tanmoy Chakraborty +32
Large language models (LLMs) are now used worldwide, yet their multimodal understanding and reasoning often degrade outside Western, high-resource settings. We propose MMA-ASIA, a…
Prompt-Guided Turn-Taking Prediction
Koji Inoue, Mikey Elmers, Yahui Fu +4
Turn-taking prediction models are essential components in spoken dialogue systems and conversational robots. Recent approaches leverage transformer-based architectures to predict s…
Triadic Multi-party Voice Activity Projection for Turn-taking in Spoken Dialogue Systems
Mikey Elmers, Koji Inoue, Divesh Lala +1
Turn-taking is a fundamental component of spoken dialogue, however conventional studies mostly involve dyadic settings. This work focuses on applying voice activity projection (VAP…
An LLM Benchmark for Addressee Recognition in Multi-modal Multi-party Dialogue
Koji Inoue, Divesh Lala, Mikey Elmers +2
Handling multi-party dialogues represents a significant step for advancing spoken dialogue systems, necessitating the development of tasks specific to multi-party interactions. To…
Why Do We Laugh? Annotation and Taxonomy Generation for Laughable Contexts in Spontaneous Text Conversation
Koji Inoue, Mikey Elmers, Divesh Lala +1
Laughter serves as a multifaceted communicative signal in human interaction, yet its identification within dialogue presents a significant challenge for conversational AI systems.…
Human-Like Embodied AI Interviewer: Employing Android ERICA in Real International Conference
Zi Haur Pang, Yahui Fu, Divesh Lala +3
This paper introduces the human-like embodied AI interviewer which integrates android robots equipped with advanced conversational capabilities, including attentive listening, conv…
Analysis and Detection of Differences in Spoken User Behaviors between Autonomous and Wizard-of-Oz Systems
Mikey Elmers, Koji Inoue, Divesh Lala +2
This study examined users' behavioral differences in a large corpus of Japanese human-robot interactions, comparing interactions between a tele-operated robot and an autonomous dia…
Yeah, Un, Oh: Continuous and Real-time Backchannel Prediction with Fine-tuning of Voice Activity Projection
Koji Inoue, Divesh Lala, Gabriel Skantze +1
In human conversations, short backchannel utterances such as "yeah" and "oh" play a crucial role in facilitating smooth and engaging dialogue. These backchannels signal attentivene…
Enhancing Long-term RAG Chatbots with Psychological Models of Memory Importance and Forgetting
Ryuichi Sumida, Koji Inoue, Tatsuya Kawahara
While Retrieval-Augmented Generation (RAG) has shown promise in enhancing long-term conversations, the increasing memory load as conversations progress degrades retrieval accuracy.…
Multilingual Turn-taking Prediction Using Voice Activity Projection
Koji Inoue, Bing'er Jiang, Erik Ekstedt +2
This paper investigates the application of voice activity projection (VAP), a predictive turn-taking model for spoken dialogue, on multilingual data, encompassing English, Mandarin…
Evaluation of a semi-autonomous attentive listening system with takeover prompting
Haruki Kawai, Divesh Lala, Koji Inoue +2
The handling of communication breakdowns and loss of engagement is an important aspect of spoken dialogue systems, particularly for chatting systems such as attentive listening, wh…
Acknowledgment of Emotional States: Generating Validating Responses for Empathetic Dialogue
Zi Haur Pang, Yahui Fu, Divesh Lala +3
In the realm of human-AI dialogue, the facilitation of empathetic responses is important. Validation is one of the key communication techniques in psychology, which entails recogni…
An Analysis of User Behaviors for Objectively Evaluating Spoken Dialogue Systems
Koji Inoue, Divesh Lala, Keiko Ochi +2
Establishing evaluation schemes for spoken dialogue systems is important, but it can also be challenging. While subjective evaluations are commonly used in user experiments, object…
Real-time and Continuous Turn-taking Prediction Using Voice Activity Projection
Koji Inoue, Bing'er Jiang, Erik Ekstedt +2
A demonstration of a real-time and continuous turn-taking prediction system is presented. The system is based on a voice activity projection (VAP) model, which directly maps dialog…
Towards Objective Evaluation of Socially-Situated Conversational Robots: Assessing Human-Likeness through Multimodal User Behaviors
Koji Inoue, Divesh Lala, Keiko Ochi +2
This paper tackles the challenging task of evaluating socially situated conversational robots and presents a novel objective evaluation approach that relies on multimodal user beha…
Reasoning before Responding: Integrating Commonsense-based Causality Explanation for Empathetic Response Generation
Yahui Fu, Koji Inoue, Chenhui Chu +1
Recent approaches to empathetic response generation try to incorporate commonsense knowledge or reasoning about the causes of emotions to better understand the user's experiences a…