19 citations · 56 across the 36 of their papers we have counts for
14 papers · 1 filter
Entropy-based Coarse and Compressed Semantic Speech Representation Learning
Jialong Zuo, Guangyan Zhang, Minghui Fang +5
Discrete speech representation learning has recently attracted increasing interest in both acoustic and semantic modeling. Existing approaches typically encode 16 kHz waveforms int…
TAP: Parameter-efficient Task-Aware Prompting for Adverse Weather Removal
Hanting Wang, Shengpeng Ji, Shulei Wang +4
Image restoration under adverse weather conditions has been extensively explored, leading to numerous high-performance methods. In particular, recent advances in All-in-One approac…
Open-set Cross Modal Generalization via Multimodal Unified Representation
Hai Huang, Yan Xia, Shulei Wang +6
This paper extends Cross Modal Generalization (CMG) to open-set environments by proposing the more challenging Open-set Cross Modal Generalization (OSCMG) task. This task evaluates…
Rhythm Controllable and Efficient Zero-Shot Voice Conversion via Shortcut Flow Matching
Jialong Zuo, Shengpeng Ji, Minghui Fang +7
Zero-Shot Voice Conversion (VC) aims to transform the source speaker's timbre into an arbitrary unseen one while retaining speech content. Most prior work focuses on preserving the…
IRBridge: Solving Image Restoration Bridge with Pre-trained Generative Diffusion Models
Hanting Wang, Tao Jin, Wang Lin +4
Bridge models in image restoration construct a diffusion process from degraded to clear images. However, existing methods typically require training a bridge model from scratch for…
T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback
Zehan Wang, Ke Lei, Chen Zhu +8
Text-to-audio (T2A) generation has achieved remarkable progress in generating a variety of audio outputs from language prompts. However, current state-of-the-art T2A models still s…
WavReward: Spoken Dialogue Models With Generalist Reward Evaluators
Shengpeng Ji, Tianle Liang, Yangzhuo Li +11
End-to-end spoken dialogue models such as GPT-4o-audio have recently garnered significant attention in the speech domain. However, the evaluation of spoken dialogue models' convers…
Astrea: A MOE-based Visual Understanding Model with Progressive Alignment
Xiaoda Yang, JunYu Lu, Hongshun Qiu +12
Vision-Language Models (VLMs) based on Mixture-of-Experts (MoE) architectures have emerged as a pivotal paradigm in multimodal understanding, offering a powerful framework for inte…
MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesis
Ziyue Jiang, Yi Ren, Ruiqi Li +11
While recent zero-shot text-to-speech (TTS) models have significantly improved speech quality and expressiveness, mainstream systems still suffer from issues related to speech-text…
InSerter: Speech Instruction Following with Unsupervised Interleaved Pre-training
Dingdong Wang, Jin Xu, Ruihang Chu +6
Recent advancements in speech large language models (SpeechLLMs) have attracted considerable attention. Nonetheless, current methods exhibit suboptimal performance in adhering to s…
UniCodec: Unified Audio Codec with Single Domain-Adaptive Codebook
Yidi Jiang, Qian Chen, Shengpeng Ji +6
The emergence of audio language models is empowered by neural audio codecs, which establish critical mappings between continuous waveforms and discrete tokens compatible with langu…
WavRAG: Audio-Integrated Retrieval Augmented Generation for Spoken Dialogue Models
Yifu Chen, Shengpeng Ji, Haoxiao Wang +5
Retrieval Augmented Generation (RAG) has gained widespread adoption owing to its capacity to empower large language models (LLMs) to integrate external knowledge. However, existing…
Enhancing Expressive Voice Conversion with Discrete Pitch-Conditioned Flow Matching Model
Jialong Zuo, Shengpeng Ji, Minghui Fang +8
This paper introduces PFlow-VC, a conditional flow matching voice conversion model that leverages fine-grained discrete pitch tokens and target speaker prompt information for expre…
OmniChat: Enhancing Spoken Dialogue Systems with Scalable Synthetic Data for Diverse Scenarios
Xize Cheng, Dongjie Fu, Xiaoda Yang +11
With the rapid development of large language models, researchers have created increasingly advanced spoken dialogue systems that can naturally converse with humans. However, these…