12 papers
The NVVSpeech Challenge 2026: Overview and Lessons for Speech Understanding and Generation with Nonverbal Vocalizations
Liumeng Xue, Yuang Cao, Jingbin Hu +12
Nonverbal vocalizations (NVVs), such as laughter and sighs, convey meaning alongside words. NVV-aware speech systems must recognize and generate these events while preserving lexic…
Long-Tail Rebalancing for Non-Verbal Vocalization-Aware ASR: A Track 1 System for the NVVSpeech Challenge
Shangyue Jia, Jingru Ma, Yangzhuo Li +9
Non-verbal vocalizations (NVVs) carry important paralinguistic information but are often omitted by conventional automatic speech recognition (ASR) systems. The ISCSLP NVVSpeech Ch…
Bridging Data, Reasoning, and Alignment: A Unified Framework for Context-Aware Instruction-Following TTS
Jingbin Hu, Luyu Wang, Wenjie Tian +7
The ISCSLP 2026 CoT-TTS Challenge requires TTS systems to generate Chain-of-Thought (CoT) reasoning from dialogue history before synthesizing contextually appropriate speech. While…
Modeling, Scaling, and Decoding: Optimizing Controllable Speech Generation with Nonverbal Vocalizations
Ziyu Zhang, Yun Chen, Taihui Wang +5
Controllable synthesis of nonverbal vocalizations (NVVs) is es- sential for natural and expressive speech, but remains challeng- ing due to their acoustic diversity and imbalanced…
Preference Optimization with LALM Feedback for Continuous Autoregressive Non-Verbal Vocalization Generation
Jingbin Hu, Qirui Zhan, Yuang Cao +8
We propose a preference optimization framework with Large Audio-Language Model (LALM) feedback for controllable non-verbal vocalization (NVV) generation in continuous autoregressiv…
SpeechAnnotator: A Context-Aware Multi-Agent Framework and Benchmark for Multidimensional Speech Annotation
Qirui Zhan, Shuiyuan Wang, Jingbin Hu +10
Recent controllable speech generation requires training data with fine-grained annotations of speaker traits, prosody, emotion, paralinguistic cues, acoustic scenes, and context. E…
HumDial-EIBench: A Human-Recorded Multi-Turn Emotional Intelligence Benchmark for Audio Language Models
Shuiyuan Wang, Zhixian Zhao, Hongfei Xue +5
Evaluating the emotional intelligence (EI) of audio language models (ALMs) is critical. However, existing benchmarks mostly rely on synthesized speech, are limited to single-turn i…
MINT-Bench: A Comprehensive Multilingual Benchmark for Instruction-Following Text-to-Speech
Huakang Chen, Jingbin Hu, Liumeng Xue +12
Instruction-following text-to-speech (TTS) has emerged as an important capability for controllable and expressive speech generation, yet its evaluation remains underdeveloped due t…
Audio-Cogito: Towards Deep Audio Reasoning in Large Audio Language Models
Longhao Li, Hongjie Chen, Zehan Li +5
Recent advances in reasoning models have driven significant progress in text and multimodal domains, yet audio reasoning remains relatively limited. Only a few Large Audio Language…
HiStyle: Hierarchical Style Embedding Predictor for Text-Prompt-Guided Controllable Speech Synthesis
Ziyu Zhang, Hanzhao Li, Jingbin Hu +2
Controllable speech synthesis refers to the precise control of speaking style by manipulating specific prosodic and paralinguistic attributes, such as gender, volume, speech rate,…
CosyAudio: Improving Audio Generation with Confidence Scores and Synthetic Captions
Xinfa Zhu, Wenjie Tian, Xinsheng Wang +4
Text-to-Audio (TTA) generation is an emerging area within AI-generated content (AIGC), where audio is created from natural language descriptions. Despite growing interest, developi…
FleSpeech: Flexibly Controllable Speech Generation with Various Prompts
Hanzhao Li, Yuke Li, Xinsheng Wang +4
Controllable speech generation methods typically rely on single or fixed prompts, hindering creativity and flexibility. These limitations make it difficult to meet specific user ne…