10 citations · 21 across the 22 of their papers we have counts for
25 papers
Multimodal Temporal Modeling for Continuous Group Emotion Recognition in Multi-party Dialogues
Soma Iwata, Koji Inoue, Muyun Wu +3
To realize natural behavior in dialogue agents in multi-party dialogue scenarios, it is important to understand group emotion such as valence and arousal as a whole. Most prior wor…
TEIDAN: A Multilingual Multiparty Dialogue Corpus
Taiga Mori, Koji Inoue, Mikey Elmers +2
Multi-party interaction is a central setting for human communication and a necessary target for human-agent interaction systems that must participate in group conversation. Yet ava…
Human-robot conversation with multiple participants in noisy public spaces
Divesh Lala, Yogeeswaran Muthukumaran, Vincent Fernandes +10
For noisy real-world environments such as those in open public spaces, spoken dialogue systems for both autonomous robots and avatars should be carefully designed to provide enhanc…
Real-time Generation of Listener Nodding via Prediction of Kinematic Parameters for Avatar Dialogue Systems
Kazushi Kato, Koji Inoue, Taiga Mori +2
In human dialogue, we achieve smooth communication by expressing nonverbal cues such as eye contact, nodding, and facial expressions with precise timing. It is expected for convers…
On the Structure of Address in Multi-Party Dialogue: From Discrete Labels to Continuous Levels
Taiga Mori, Koji Inoue, Divesh Lala +1
In multi-party dialogues between a dialogue system and multiple users, identifying to whom an utterance is addressed is a key challenge. Prior work has typically treated addressee…
Multilingual and Continuous Backchannel Prediction: A Cross-lingual Study
Koji Inoue, Mikey Elmers, Yahui Fu +5
We present a multilingual, continuous backchannel prediction model for Japanese, English, and Chinese, and use it to investigate cross-linguistic timing behavior. The model is Tran…
Why Report Failed Interactions With Robots?! Towards Vignette-based Interaction Quality
Agnes Axelsson, Merle Reimann, Ronald Cumbal +2
Although the quality of human-robot interactions has improved with the advent of LLMs, there are still various factors that cause systems to be sub-optimal when compared to human-h…
Real-time Generation of Various Types of Nodding for Avatar Attentive Listening System
Kazushi Kato, Koji Inoue, Divesh Lala +2
In human dialogue, nonverbal information such as nodding and facial expressions is as crucial as verbal information, and spoken dialogue systems are also expected to express such n…
Prompt-Guided Turn-Taking Prediction
Koji Inoue, Mikey Elmers, Yahui Fu +4
Turn-taking prediction models are essential components in spoken dialogue systems and conversational robots. Recent approaches leverage transformer-based architectures to predict s…
Triadic Multi-party Voice Activity Projection for Turn-taking in Spoken Dialogue Systems
Mikey Elmers, Koji Inoue, Divesh Lala +1
Turn-taking is a fundamental component of spoken dialogue, however conventional studies mostly involve dyadic settings. This work focuses on applying voice activity projection (VAP…
Does the Appearance of Autonomous Conversational Robots Affect User Spoken Behaviors in Real-World Conference Interactions?
Zi Haur Pang, Yahui Fu, Divesh Lala +3
We investigate the impact of robot appearance on users' spoken behavior during real-world interactions by comparing a human-like android, ERICA, with a less anthropomorphic humanoi…
An LLM Benchmark for Addressee Recognition in Multi-modal Multi-party Dialogue
Koji Inoue, Divesh Lala, Mikey Elmers +2
Handling multi-party dialogues represents a significant step for advancing spoken dialogue systems, necessitating the development of tasks specific to multi-party interactions. To…
Why Do We Laugh? Annotation and Taxonomy Generation for Laughable Contexts in Spontaneous Text Conversation
Koji Inoue, Mikey Elmers, Divesh Lala +1
Laughter serves as a multifaceted communicative signal in human interaction, yet its identification within dialogue presents a significant challenge for conversational AI systems.…
Human-Like Embodied AI Interviewer: Employing Android ERICA in Real International Conference
Zi Haur Pang, Yahui Fu, Divesh Lala +3
This paper introduces the human-like embodied AI interviewer which integrates android robots equipped with advanced conversational capabilities, including attentive listening, conv…
Analysis and Detection of Differences in Spoken User Behaviors between Autonomous and Wizard-of-Oz Systems
Mikey Elmers, Koji Inoue, Divesh Lala +2
This study examined users' behavioral differences in a large corpus of Japanese human-robot interactions, comparing interactions between a tele-operated robot and an autonomous dia…
Yeah, Un, Oh: Continuous and Real-time Backchannel Prediction with Fine-tuning of Voice Activity Projection
Koji Inoue, Divesh Lala, Gabriel Skantze +1
In human conversations, short backchannel utterances such as "yeah" and "oh" play a crucial role in facilitating smooth and engaging dialogue. These backchannels signal attentivene…
Evaluation of a semi-autonomous attentive listening system with takeover prompting
Haruki Kawai, Divesh Lala, Koji Inoue +2
The handling of communication breakdowns and loss of engagement is an important aspect of spoken dialogue systems, particularly for chatting systems such as attentive listening, wh…
Acknowledgment of Emotional States: Generating Validating Responses for Empathetic Dialogue
Zi Haur Pang, Yahui Fu, Divesh Lala +3
In the realm of human-AI dialogue, the facilitation of empathetic responses is important. Validation is one of the key communication techniques in psychology, which entails recogni…
An Analysis of User Behaviors for Objectively Evaluating Spoken Dialogue Systems
Koji Inoue, Divesh Lala, Keiko Ochi +2
Establishing evaluation schemes for spoken dialogue systems is important, but it can also be challenging. While subjective evaluations are commonly used in user experiments, object…
Towards Objective Evaluation of Socially-Situated Conversational Robots: Assessing Human-Likeness through Multimodal User Behaviors
Koji Inoue, Divesh Lala, Keiko Ochi +2
This paper tackles the challenging task of evaluating socially situated conversational robots and presents a novel objective evaluation approach that relies on multimodal user beha…
Alzheimer's Dementia Detection through Spontaneous Dialogue with Proactive Robotic Listeners
Yuanchao Li, Catherine Lai, Divesh Lala +2
As the aging of society continues to accelerate, Alzheimer's Disease (AD) has received more and more attention from not only medical but also other fields, such as computer science…
ERICA: An Empathetic Android Companion for Covid-19 Quarantine
Etsuko Ishii, Genta Indra Winata, Samuel Cahyawijaya +3
Over the past year, research in various domains, including Natural Language Processing (NLP), has been accelerated to fight against the COVID-19 pandemic, yet such research has jus…
Intelligent Conversational Android ERICA Applied to Attentive Listening and Job Interview
Tatsuya Kawahara, Koji Inoue, Divesh Lala
Following the success of spoken dialogue systems (SDS) in smartphone assistants and smart speakers, a number of communicative robots are developed and commercialized. Compared with…
Designing Precise and Robust Dialogue Response Evaluators
Tianyu Zhao, Divesh Lala, Tatsuya Kawahara
Automatic dialogue response evaluator has been proposed as an alternative to automated metrics and human evaluation. However, existing automatic evaluators achieve only moderate co…