11 papers
SemanticAudio: Audio Generation and Editing in Semantic Space
Zheqi Dai, Guangyan Zhang, Haolin He +5
In recent years, Text-to-Audio Generation has achieved remarkable progress, offering sound creators powerful tools to transform textual inspirations into vivid audio. However, exis…
One-Step Token-to-Waveform Generation with MeanFlow in Latent Space
Zheqi Dai, Guangyan Zhang, Zhen Ye +5
Neural audio codecs are central to modern LLM-based Text-to-Speech (TTS) and multimodal systems. As low-bitrate semantic codecs gain prominence, the Token-to-Waveform (Token2Wav) d…
Which Speech Representation Better Matches Text-Native Reasoning? A Study of Speech-Text Alignment on Frame Rate and Representation
Zhen Ye, Xu Tan, Yiming Li +10
Spoken dialogue models typically start from text LLM backbones, yet reasoning often degrades when conditioning on speech instead of text. We attribute part of this modality gap to…
Talker-T2AV: Joint Talking Audio-Video Generation with Autoregressive Diffusion Modeling
Zhen Ye, Xu Tan, Aoxiong Yin +8
Joint audio-video generation models have shown that unified generation yields stronger cross-modal coherence than cascaded approaches. However, existing models couple modalities th…
AudioRole: An Audio Dataset for Character Role-Playing in Large Language Models
Wenyu Li, Xiaoqi Jiao, Yi Chang +2
The creation of high-quality multimodal datasets remains fundamental for advancing role-playing capabilities in large language models (LLMs). While existing works predominantly foc…
MSR-Codec: A Low-Bitrate Multi-Stream Residual Codec for High-Fidelity Speech Generation with Information Disentanglement
Jingyu Li, Guangyan Zhang, Zhen Ye +1
Audio codecs are a critical component of modern speech generation systems. This paper introduces a low-bitrate, multi-scale residual codec that encodes speech into four distinct st…