Giving Robots a Voice: Human-in-the-Loop Voice Creation and open-ended Labeling

Abstract

Speech is a natural interface for humans to interact with robots. Yet, aligning a robot's voice to its appearance is challenging due to the rich vocabulary of both modalities. Previous research has explored a few labels to describe robots and tested them on a limited number of robots and existing voices. Here, we develop a robot-voice creation tool followed by large-scale behavioral human experiments (N=2,505). First, participants collectively tune robotic voices to match 175 robot images using an adaptive human-in-the-loop pipeline. Then, participants describe their impression of the robot or their matched voice using another human-in-the-loop paradigm for open-ended labeling. The elicited taxonomy is then used to rate robot attributes and to predict the best voice for an unseen robot. We offer a web interface to aid engineers in customizing robot voices, demonstrating the synergy between cognitive science and machine learning for engineering tools.

Accepted to CHI 2024, May 11 to 16, 2024, Honolulu, HI, USA

References in corpus (6)

• Learning Transferable Visual Models From Natural Language Supervision
• Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
• Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search
• A Survey on Neural Speech Synthesis
• Developing a Personality Model for Speech-based Conversational Agents Using the Psycholexical Approach
• Exploring emotional prototypes in a high dimensional TTS latent space