10 citations · 24 across the 34 of their papers we have counts for
38 papers
Multimodal Temporal Modeling for Continuous Group Emotion Recognition in Multi-party Dialogues
Soma Iwata, Koji Inoue, Muyun Wu +3
To realize natural behavior in dialogue agents in multi-party dialogue scenarios, it is important to understand group emotion such as valence and arousal as a whole. Most prior wor…
TEIDAN: A Multilingual Multiparty Dialogue Corpus
Taiga Mori, Koji Inoue, Mikey Elmers +2
Multi-party interaction is a central setting for human communication and a necessary target for human-agent interaction systems that must participate in group conversation. Yet ava…
Human-robot conversation with multiple participants in noisy public spaces
Divesh Lala, Yogeeswaran Muthukumaran, Vincent Fernandes +10
For noisy real-world environments such as those in open public spaces, spoken dialogue systems for both autonomous robots and avatars should be carefully designed to provide enhanc…
CultureConverse: A Multilingual Multi-turn Simulation Harness for Culturally Grounded Assistance in East and Southeast Asia
Bryan Chen Zhengyu Tan, Weihua Zheng, Thong T. Doan +30
Current cultural evaluations for large language models (LLMs) often reduce culture to single-turn factual recall via MCQs, failing to capture a common use case: users seeking pract…
MemUse: Moving Memory Evaluation from Direct QA to Natural Integration in Long-Term Human-AI Conversation
Ryuichi Sumida, Koji Inoue, Tatsuya Kawahara
Memory systems for conversational LLMs are conventionally evaluated by direct, fact-seeking questions about prior dialogue (Direct QA): can the model recall fact X from a prior con…
Does Listening Matter? Backchanneling and Nodding in AI Clone
Koji Inoue, Kazushi Kato, Tatsuya Kawahara +1
AI clones that imitate a specific person typically reproduce what the person says and how they sound, but not how they listen. We investigate whether adding multimodal listening be…
Real-time Generation of Listener Nodding via Prediction of Kinematic Parameters for Avatar Dialogue Systems
Kazushi Kato, Koji Inoue, Taiga Mori +2
In human dialogue, we achieve smooth communication by expressing nonverbal cues such as eye contact, nodding, and facial expressions with precise timing. It is expected for convers…
Low-Latency Turn-Taking via Context-Aware Preface Generation in a Real-World Dialogue Robot
Yuki Okafuji, Koji Inoue, Yoshiki Ohira
Large language model (LLM)-based dialogue systems suffer response delays because generation begins only after final speech recognition. While fixed fillers are a workaround, they b…
On the Structure of Address in Multi-Party Dialogue: From Discrete Labels to Continuous Levels
Taiga Mori, Koji Inoue, Divesh Lala +1
In multi-party dialogues between a dialogue system and multiple users, identifying to whom an utterance is addressed is a key challenge. Prior work has typically treated addressee…
Memory-Driven Self-Disclosure and Relational Turning Points: A Longitudinal Multimodal Study of Human-AI Interaction
Ryuichi Sumida, Mao Saeki, Masaki Eguchi +4
As conversational AI systems are designed for repeated use, a central question is how a series of interactions becomes a relationship. We present a longitudinal multimodal study of…
I Understand How You Feel: Enhancing Deeper Emotional Support Through Multilingual Emotional Validation in Dialogue System
Zi Haur Pang, Yahui Fu, Koji Inoue +1
Emotional validation - explicitly acknowledging that a user's feelings make sense - has proven therapeutic value but has received little computational attention. Emotional validati…
Toward Signing Activity Projection in Sign Language Interaction
Takao Obi, Wang Yusong, Koji Inoue +1
Social robots must interact robustly not only with users assumed by speech-centered systems but also with diverse users whose communication relies on different modalities, e.g., si…
Emotion Transcription in Conversation: A Benchmark for Capturing Subtle and Complex Emotional States through Natural Language
Yoshiki Tanaka, Ryuichi Uehara, Koji Inoue +1
Emotion Recognition in Conversation (ERC) is critical for enabling natural human-machine interactions. However, existing methods predominantly employ categorical or dimensional emo…
Multilingual and Continuous Backchannel Prediction: A Cross-lingual Study
Koji Inoue, Mikey Elmers, Yahui Fu +5
We present a multilingual, continuous backchannel prediction model for Japanese, English, and Chinese, and use it to investigate cross-linguistic timing behavior. The model is Tran…
MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation
Weihua Zheng, Zhengyuan Liu, Tanmoy Chakraborty +32
Large language models (LLMs) are now used worldwide, yet their multimodal understanding and reasoning often degrade outside Western, high-resource settings. We propose MMA-ASIA, a…
Real-time Generation of Various Types of Nodding for Avatar Attentive Listening System
Kazushi Kato, Koji Inoue, Divesh Lala +2
In human dialogue, nonverbal information such as nodding and facial expressions is as crucial as verbal information, and spoken dialogue systems are also expected to express such n…
Prompt-Guided Turn-Taking Prediction
Koji Inoue, Mikey Elmers, Yahui Fu +4
Turn-taking prediction models are essential components in spoken dialogue systems and conversational robots. Recent approaches leverage transformer-based architectures to predict s…
Triadic Multi-party Voice Activity Projection for Turn-taking in Spoken Dialogue Systems
Mikey Elmers, Koji Inoue, Divesh Lala +1
Turn-taking is a fundamental component of spoken dialogue, however conventional studies mostly involve dyadic settings. This work focuses on applying voice activity projection (VAP…
Does the Appearance of Autonomous Conversational Robots Affect User Spoken Behaviors in Real-World Conference Interactions?
Zi Haur Pang, Yahui Fu, Divesh Lala +3
We investigate the impact of robot appearance on users' spoken behavior during real-world interactions by comparing a human-like android, ERICA, with a less anthropomorphic humanoi…
A Noise-Robust Turn-Taking System for Real-World Dialogue Robots: A Field Experiment
Koji Inoue, Yuki Okafuji, Jun Baba +3
Turn-taking is a crucial aspect of human-robot interaction, directly influencing conversational fluidity and user engagement. While previous research has explored turn-taking model…
An LLM Benchmark for Addressee Recognition in Multi-modal Multi-party Dialogue
Koji Inoue, Divesh Lala, Mikey Elmers +2
Handling multi-party dialogues represents a significant step for advancing spoken dialogue systems, necessitating the development of tasks specific to multi-party interactions. To…
Why Do We Laugh? Annotation and Taxonomy Generation for Laughable Contexts in Spontaneous Text Conversation
Koji Inoue, Mikey Elmers, Divesh Lala +1
Laughter serves as a multifaceted communicative signal in human interaction, yet its identification within dialogue presents a significant challenge for conversational AI systems.…
Human-Like Embodied AI Interviewer: Employing Android ERICA in Real International Conference
Zi Haur Pang, Yahui Fu, Divesh Lala +3
This paper introduces the human-like embodied AI interviewer which integrates android robots equipped with advanced conversational capabilities, including attentive listening, conv…
Analysis and Detection of Differences in Spoken User Behaviors between Autonomous and Wizard-of-Oz Systems
Mikey Elmers, Koji Inoue, Divesh Lala +2
This study examined users' behavioral differences in a large corpus of Japanese human-robot interactions, comparing interactions between a tele-operated robot and an autonomous dia…