14 papers
Post-Training Zero-Shot TTS for Fine-Grained Emotion and Duration Control via Natural Language
Lianru Gao, Yujie Guo, Yong Qin
Audiobook narration, conversational agents, and audiovisual dubbing require fine-grained control over emotion and pacing within an utterance, yet many text-to-speech (TTS) systems…
A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation
Liuyi Wang, Kai Sheng, Zongtao He +8
Navigation is a fundamental capability of autonomous systems, yet most existing approaches rely on highly structured models and strong prior assumptions, limiting their robustness…
H-SAGE: Holistic Speaker-Aware Guided Experts for MoE-based Multi-Talker ASR
Yujie Guo, Jiaming Zhou, Yuhang Jia +2
Multi-talker Automatic Speech Recognition (MTASR) faces significant challenges in accurately transcribing overlapping speech, particularly under complex high-overlap conditions. Wh…
CosyEdit2: Speech-Editing-Oriented Reinforcement Learning Unlocks Better Zero-Shot TTS
Junyang Chen, Yuhang Jia, Hui Wang +3
Speech editing and zero-shot Text-to-Speech (TTS) share a similar generative foundation conditioned on speech prompts, yet speech editing demands far stricter local acoustic consis…
P2DNav: Panorama-to-Downview Reasoning for Zero-shot Vision-and-Language Navigation
Kai Sheng, Liuyi Wang, Haojie Dai +5
Vision-and-language navigation (VLN) requires an embodied agent to ground natural-language instructions into executable navigation actions in unseen environments. Existing zero-sho…
CosyEdit: Unlocking End-to-End Speech Editing Capability from Zero-Shot Text-to-Speech Models
Junyang Chen, Yuhang Jia, Hui Wang +2
Automatic speech editing aims to modify spoken content based on textual instructions, yet traditional cascade systems rely on explicit temporal alignment and complex preprocessing.…
SpeechLLM-as-Judges: Towards General and Interpretable Speech Quality Evaluation
Hui Wang, Jinghua Zhao, Yifan Yang +9
Generative speech technologies are progressing rapidly, but evaluating the perceptual quality of synthetic speech remains a core challenge. Existing methods typically rely on scala…
AudioEval: Automatic Dual-Perspective and Multi-Dimensional Evaluation of Text-to-Audio-Generation
Hui Wang, Jinghua Zhao, Junyang Cheng +5
Text-to-audio (TTA) generation is advancing rapidly, but evaluation remains challenging because human listening studies are expensive and existing automatic metrics capture only li…
Position: Towards Responsible Evaluation for Text-to-Speech
Yifan Yang, Hui Wang, Bing Han +4
Recent advances in text-to-speech (TTS) technology have enabled systems to generate speech that is often indistinguishable from human speech, bringing benefits to accessibility, co…
Omni-CLST: Error-aware Curriculum Learning with guided Selective chain-of-Thought for audio question answering
Jinghua Zhao, Hang Su, Lichun Fan +4
With the rapid progress of large audio-language models (LALMs), audio question answering (AQA) has emerged as a challenging task requiring both fine-grained audio understanding and…
Interpretable Audio Editing Evaluation via Chain-of-Thought Difference-Commonality Reasoning with Multimodal LLMs
Yuhang Jia, Xu Zhang, Yang Chen +3
Automatic mean opinion score (MOS) prediction serves as a principled alternative to both subjective listening tests and objective metrics, providing scalable and consistent audio e…
GLAD: Global-Local Aware Dynamic Mixture-of-Experts for Multi-Talker ASR
Yujie Guo, Jiaming Zhou, Yuhang Jia +2
End-to-end multi-talker automatic speech recognition (MTASR) faces significant challenges in accurately transcribing overlapping speech. A critical bottleneck is that speaker-speci…
Towards Automatic Evaluation and High-Quality Pseudo-Parallel Dataset Construction for Audio Editing: A Human-in-the-Loop Method
Yuhang Jia, Hui Wang, Xin Nie +3
Audio editing aims to manipulate audio content based on textual descriptions, supporting tasks such as adding, removing, or replacing audio events. Despite recent progress, the lac…
From Contrast to Commonality: Audio Commonality Captioning for Enhanced Audio-Text Cross-modal Understanding in Multimodal LLMs
Yuhang Jia, Xu Zhang, Yujie Guo +2
Audio Captioning (AC) plays a pivotal role in enhancing audio-text cross-modal understanding during the pretraining and finetuning of Multimodal LLMs (MLLMs). To strengthen this al…