18 citations · 101 across the 75 of their papers we have counts for
16 papers · 1 filter
X-Talk: On the Underestimated Potential of Modular Speech-to-Speech Dialogue System
Zhanxun Liu, Yifan Duan, Mengmeng Wang +15
We present X-Talk, an open-source framework that champions a decoupled, modular design for LLM-driven speech-to-speech (S2S) systems. While the dominant trend favors end-to-end (E2…
Task Vector in TTS: Toward Emotionally Expressive Dialectal Speech Synthesis
Pengchao Feng, Yao Xiao, Ziyang Ma +5
Recent advances in text-to-speech (TTS) have yielded remarkable improvements in naturalness and intelligibility. Building on these achievements, research has increasingly shifted t…
ISA-Bench: Benchmarking Instruction Sensitivity for Large Audio Language Models
Bohan Li, Wenbin Huang, Yuhang Qiu +7
Large Audio Language Models (LALMs), which couple acoustic perception with large language models (LLMs) to extract and understand diverse information from audio, have attracted int…
UltraVoice: Scaling Fine-Grained Style-Controlled Speech Conversations for Spoken Dialogue Models
Wenming Tu, Guanrou Yang, Ruiqi Yan +6
Spoken dialogue models currently lack the ability for fine-grained speech style control, a critical capability for human-like interaction that is often overlooked in favor of purel…
SAC: Neural Speech Codec with Semantic-Acoustic Dual-Stream Quantization
Wenxi Chen, Xinsheng Wang, Ruiqi Yan +9
Speech codecs that convert continuous speech signals into discrete tokens have become essential for speech language models. However, existing codecs struggle to balance high-qualit…
Accelerating Flow-Matching-Based Text-to-Speech via Empirically Pruned Step Sampling
Qixi Zheng, Yushen Chen, Zhikang Niu +4
Flow-matching-based text-to-speech (TTS) models, such as Voicebox, E2 TTS, and F5-TTS, have attracted significant attention in recent years. These models require multiple sampling…
MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix
Ziyang Ma, Yinghao Ma, Yanqiao Zhu +31
We introduce MMAR, a new benchmark designed to evaluate the deep reasoning capabilities of Audio-Language Models (ALMs) across massive multi-disciplinary tasks. MMAR comprises 1,00…
Towards Flow-Matching-based TTS without Classifier-Free Guidance
Yuzhe Liang, Wenzhe Liu, Chunyu Qiang +7
Flow matching has demonstrated strong generative capabilities and has become a core component in modern Text-to-Speech (TTS) systems. To ensure high-quality speech synthesis, Class…
Enhancing Speech-to-Speech Dialogue Modeling with End-to-End Retrieval-Augmented Generation
Pengchao Feng, Ziyang Ma, Wenxi Chen +4
End-to-end speech-to-speech (S2S) dialogue systems have recently garnered increasing research attention for their lower latency and more natural integration of nonverbal cues such…
MER 2025: When Affective Computing Meets Large Language Models
Zheng Lian, Rui Liu, Kele Xu +15
MER2025 is the third year of our MER series of challenges, aiming to bring together researchers in the affective computing community to explore emerging trends and future direction…
EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Prompting
Guanrou Yang, Chen Yang, Qian Chen +12
Human speech goes beyond the mere transfer of information; it is a profound exchange of emotions and a connection between individuals. While Text-to-Speech (TTS) models have made h…
Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens
Xinsheng Wang, Mingqi Jiang, Ziyang Ma +22
Recent advancements in large language models (LLMs) have driven significant progress in zero-shot text-to-speech (TTS) synthesis. However, existing foundation models rely on multi-…
YuE: Scaling Open Foundation Models for Long-Form Music Generation
Ruibin Yuan, Hanfeng Lin, Shuyue Guo +55
We tackle the task of long-form music generation--particularly the challenging \textbf{lyrics-to-song} problem--by introducing YuE, a family of open foundation models based on the…
URO-Bench: Towards Comprehensive Evaluation for End-to-End Spoken Dialogue Models
Ruiqi Yan, Xiquan Li, Wenxi Chen +5
Recent advances in large language models (LLMs) have driven significant progress in end-to-end spoken dialogue models (SDMs). In contrast to text-based LLMs, the evaluation framewo…
Audio-CoT: Exploring Chain-of-Thought Reasoning in Large Audio Language Model
Ziyang Ma, Zhuo Chen, Yuping Wang +2
Large Audio-Language Models (LALMs) have demonstrated remarkable performance in tasks involving audio perception and understanding, such as speech recognition and audio captioning.…
MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantization
Haina Zhu, Yizhi Zhou, Hangting Chen +6
Recent years have witnessed the success of foundation models pre-trained with self-supervised learning (SSL) in various music informatics understanding tasks, including music taggi…