5 citations · 6 across the 12 of their papers we have counts for
12 papers
One-Step Token-to-Waveform Generation with MeanFlow in Latent Space
Zheqi Dai, Guangyan Zhang, Zhen Ye +5
Neural audio codecs are central to modern LLM-based Text-to-Speech (TTS) and multimodal systems. As low-bitrate semantic codecs gain prominence, the Token-to-Waveform (Token2Wav) d…
Which Speech Representation Better Matches Text-Native Reasoning? A Study of Speech-Text Alignment on Frame Rate and Representation
Zhen Ye, Xu Tan, Yiming Li +10
Spoken dialogue models typically start from text LLM backbones, yet reasoning often degrades when conditioning on speech instead of text. We attribute part of this modality gap to…
Talker-T2AV: Joint Talking Audio-Video Generation with Autoregressive Diffusion Modeling
Zhen Ye, Xu Tan, Aoxiong Yin +8
Joint audio-video generation models have shown that unified generation yields stronger cross-modal coherence than cascaded approaches. However, existing models couple modalities th…
SemanticAudio: Audio Generation and Editing in Semantic Space
Zheqi Dai, Guangyan Zhang, Haolin He +5
In recent years, Text-to-Audio Generation has achieved remarkable progress, offering sound creators powerful tools to transform textual inspirations into vivid audio. However, exis…
Empathy Omni: Enabling Empathetic Speech Response Generation through Large Language Models
Haoyu Wang, Guangyan Zhang, Jiale Chen +3
With the development of speech large language models (speech LLMs), users can now interact directly with assistants via speech. However, most existing models only convert response…
AudioRole: An Audio Dataset for Character Role-Playing in Large Language Models
Wenyu Li, Xiaoqi Jiao, Yi Chang +2
The creation of high-quality multimodal datasets remains fundamental for advancing role-playing capabilities in large language models (LLMs). While existing works predominantly foc…
MSR-Codec: A Low-Bitrate Multi-Stream Residual Codec for High-Fidelity Speech Generation with Information Disentanglement
Jingyu Li, Guangyan Zhang, Zhen Ye +1
Audio codecs are a critical component of modern speech generation systems. This paper introduces a low-bitrate, multi-scale residual codec that encodes speech into four distinct st…
Entropy-based Coarse and Compressed Semantic Speech Representation Learning
Jialong Zuo, Guangyan Zhang, Minghui Fang +5
Discrete speech representation learning has recently attracted increasing interest in both acoustic and semantic modeling. Existing approaches typically encode 16 kHz waveforms int…
SpeechAccentLLM: A Unified Framework for Foreign Accent Conversion and Text to Speech
Zhuangfei Cheng, Guangyan Zhang, Zehai Tu +6
Foreign accent conversion (FAC) in speech processing remains a challenging task. Building on the remarkable success of large language models (LLMs) in Text-to-Speech (TTS) tasks, t…
Enhancing Expressive Voice Conversion with Discrete Pitch-Conditioned Flow Matching Model
Jialong Zuo, Shengpeng Ji, Minghui Fang +8
This paper introduces PFlow-VC, a conditional flow matching voice conversion model that leverages fine-grained discrete pitch tokens and target speaker prompt information for expre…
Recent Advances in Speech Language Models: A Survey
Wenqian Cui, Dianzhi Yu, Xiaoqi Jiao +5
Large Language Models (LLMs) have recently garnered significant attention, primarily for their capabilities in text-based interactions. However, natural human interaction often rel…
Enabling Beam Search for Language Model-Based Text-to-Speech Synthesis
Zehai Tu, Guangyan Zhang, Yiting Lu +3
Tokenising continuous speech into sequences of discrete tokens and modelling them with language models (LMs) has led to significant success in text-to-speech (TTS) synthesis. Altho…