8 papers
SonicWeave: Chunk-Routed Mixture-of-Experts for Unified Audio Scene Generation
Yunrui Cai, Xu Li, Yucheng Zhou +8
Text-conditioned general audio generation is moving beyond isolated speech, music, and sound-effect synthesis toward a single model that can compose them into controllable, coheren…
How Should LLMs Listen While Speaking? A Study of User-Stream Routing in Full-Duplex Spoken Dialogue
Hui Lu, Xueyuan Chen, Huimeng Wang +4
Full-duplex spoken dialogue requires a model to keep listening while generating its own spoken response. This is challenging for large language models (LLMs), which are designed to…
DualSpeechLM: Towards Unified Speech Understanding and Generation via Dual Speech Token Modeling with Large Language Models
Yuanyuan Wang, Dongchao Yang, Yiwen Shao +5
Extending pre-trained text Large Language Models (LLMs)'s speech understanding or generation abilities by introducing various effective speech tokens has attracted great attention…
WAKE: Watermarking Audio with Key Enrichment
Yaoxun Xu, Jianwei Yu, Hangting Chen +5
As deep learning advances in audio generation, challenges in audio security and copyright protection highlight the need for robust audio watermarking. Recent neural network-based m…
Enhancing Generalization of Speech Large Language Models with Multi-Task Behavior Imitation and Speech-Text Interleaving
Jingran Xie, Xiang Li, Hui Wang +4
Large language models (LLMs) have shown remarkable generalization across tasks, leading to increased interest in integrating speech with LLMs. These speech LLMs (SLLMs) typically u…
UniSep: Universal Target Audio Separation with Language Models at Scale
Yuanyuan Wang, Hangting Chen, Dongchao Yang +7
We propose Universal target audio Separation (UniSep), addressing the separation task on arbitrary mixtures of different types of audio. Distinguished from previous studies, UniSep…
Leveraging Chain of Thought towards Empathetic Spoken Dialogue without Corresponding Question-Answering Data
Jingran Xie, Shun Lei, Yue Yu +4
Empathetic dialogue is crucial for natural human-computer interaction, allowing the dialogue system to respond in a more personalized and emotionally aware manner, improving user s…
DrawSpeech: Expressive Speech Synthesis Using Prosodic Sketches as Control Conditions
Weidong Chen, Shan Yang, Guangzhi Li +1
Controlling text-to-speech (TTS) systems to synthesize speech with the prosodic characteristics expected by users has attracted much attention. To achieve controllability, current…