5 papers
NVV-Locator: From Transcript Tags to Acoustic Boundaries for Fine-Grained Nonverbal Vocalization Grounding
Yuang Cao, Bingshen Mu, Zhennan Lin +7
Human speech includes nonverbal vocalizations (NVVs), such as laughter, sighs, breaths, and coughs, which convey affective and interactional information. Existing approaches typica…
Towards Fine-Grained Multi-Dimensional Speech Understanding: Data Pipeline, Benchmark, and Model
Guojian Li, Zhixian Zhao, Zhennan Lin +9
While speech Large Language Models (LLMs) excel at conventional tasks like basic speech recognition, they lack fine-grained, multi-dimensional perception. This deficiency is eviden…
MINT-Bench: A Comprehensive Multilingual Benchmark for Instruction-Following Text-to-Speech
Huakang Chen, Jingbin Hu, Liumeng Xue +12
Instruction-following text-to-speech (TTS) has emerged as an important capability for controllable and expressive speech generation, yet its evaluation remains underdeveloped due t…
Self-Emotion Blended Dialogue Generation in Social Simulation Agents
Qiang Zhang, Jason Naradowsky, Yusuke Miyao
When engaging in conversations, dialogue agents in a virtual simulation environment may exhibit their own emotional states that are unrelated to the immediate conversational contex…
Mind the Gap Between Conversations for Improved Long-Term Dialogue Generation
Qiang Zhang, Jason Naradowsky, Yusuke Miyao
Knowing how to end and resume conversations over time is a natural part of communication, allowing for discussions to span weeks, months, or years. The duration of gaps between con…