5 citations · 6 across the 11 of their papers we have counts for
12 papers
VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference
Wenxu Jia, Dongjie Fu, Xize Cheng +6
Recent advancements in Speech Large Language Models have demonstrated remarkable capabilities in understanding complex audio tasks. Despite this progress, their long-context infere…
Character Beyond Speech: Leveraging Role-Playing Evaluation in Audio Large Language Models via Reinforcement Learning
Dongjie Fu, Fangming Feng, Xize Cheng +3
The rapid evolution of multimodal large models has revolutionized the simulation of diverse characters in speech dialogue systems, enabling a novel interactive paradigm. Character…
MARS-Sep: Multimodal-Aligned Reinforced Sound Separation
Zihan Zhang, Xize Cheng, Zhennan Jiang +4
Universal sound separation faces a fundamental misalignment: models optimized for low-level signal metrics often produce semantically contaminated outputs, failing to suppress perc…
T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback
Zehan Wang, Ke Lei, Chen Zhu +8
Text-to-audio (T2A) generation has achieved remarkable progress in generating a variety of audio outputs from language prompts. However, current state-of-the-art T2A models still s…
WavReward: Spoken Dialogue Models With Generalist Reward Evaluators
Shengpeng Ji, Tianle Liang, Yangzhuo Li +11
End-to-end spoken dialogue models such as GPT-4o-audio have recently garnered significant attention in the speech domain. However, the evaluation of spoken dialogue models' convers…
Unleashing the Power of Natural Audio Featuring Multiple Sound Sources
Xize Cheng, Slytherin Wang, Zehan Wang +3
Universal sound separation aims to extract clean audio tracks corresponding to distinct events from mixed audio, which is critical for artificial auditory perception. However, curr…
Multimodal Prompt Learning with Missing Modalities for Sentiment Analysis and Emotion Recognition
Zirun Guo, Tao Jin, Zhou Zhao
The development of multimodal models has significantly advanced multimodal sentiment analysis and emotion recognition. However, in real-world applications, the presence of various…
Text-to-Song: Towards Controllable Music Generation Incorporating Vocals and Accompaniment
Zhiqing Hong, Rongjie Huang, Xize Cheng +5
A song is a combination of singing voice and accompaniment. However, existing works focus on singing voice synthesis and music generation independently. Little attention was paid t…
FreeBind: Free Lunch in Unified Multimodal Space via Knowledge Fusion
Zehan Wang, Ziang Zhang, Xize Cheng +8
Unified multi-model representation spaces are the foundation of multimodal understanding and generation. However, the billions of model parameters and catastrophic forgetting probl…
Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt
Yongqi Wang, Ruofan Hu, Rongjie Huang +6
Recent singing-voice-synthesis (SVS) methods have achieved remarkable audio quality and naturalness, yet they lack the capability to control the style attributes of the synthesized…
TransFace: Unit-Based Audio-Visual Speech Synthesizer for Talking Head Translation
Xize Cheng, Rongjie Huang, Linjun Li +7
Direct speech-to-speech translation achieves high-quality results through the introduction of discrete units obtained from self-supervised learning. This approach circumvents delay…
Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifiers
Haifeng Huang, Yilun Chen, Zehan Wang +8
Recent advancements in 3D Large Language Models (LLMs) have demonstrated promising capabilities for 3D scene understanding. However, previous methods exhibit deficiencies in genera…