12 papers
EmoTrace-Bench: Diagnosing Emotional Expression in Text-to-Video Generation through Controlled Cues and Intensity
Yeheng Wang, Liudan Shi, Yi Zheng +5
Text-to-video (T2V) models are increasingly used to generate videos with targeted emotional expressions, yet existing evaluation methods mainly rely on holistic video-text alignmen…
UniStream: Multi-Expert Residual Vector Quantization for 48 kHz Causal Streaming Audio Coding
Mingyu Zhao, Zhiyong Wu
We present UniStream, a fully causal 48 kHz neural audio codec for streaming speech, music, and environmental sounds. At its core is Multi-Expert Residual Vector Quantization (ME-R…
BAMU: Bitstream-Aware Marginal-Utility Allocation for Frozen Pretrained Neural Speech Codecs
Mingyu Zhao, Zijian Lin, Yutang Feng +6
Pretrained neural speech codecs typically use a fixed residual vector quantization (RVQ) depth for all frames, ignoring temporal variation in quantization difficulty. We propose BA…
Qwen-Music Technical Report
Jin Xu, Kangdi Wang, Ruibin Yuan +24
We introduce Qwen-Music, a music generation model that produces high-fidelity songs with complete vocals. It supports text-to-music generation from descriptions, lyrics, and musica…
SPG-Codec: Exploring the Role and Boundaries of Semantic Priors in Ultra-Low-Bitrate Neural Speech Coding
Mingyu Zhao, Zijian Lin, Kun Wei +1
Conventional neural speech codecs suffer from severe intelligibility degradation at ultra-low bitrates, where the bottleneck transitions from acoustic distortion to semantic loss.…
SongBench: A Fine-Grained Multi-Aspect Benchmark for Song Quality Assessment
Dapeng Wu, Shun Lei, Wei Tan +5
Recent advancements in Text-to-Song generation have enabled realistic musical content production, yet existing evaluation benchmarks lack the professional granularity to capture mu…
DAFMSVC: One-Shot Singing Voice Conversion with Dual Attention Mechanism and Flow Matching
Wei Chen, Binzhu Sha, Dan Luo +4
Singing Voice Conversion (SVC) transfers a source singer's timbre to a target while keeping melody and lyrics. The key challenge in any-to-any SVC is adapting unseen speaker timbre…
DiffCSS: Diverse and Expressive Conversational Speech Synthesis with Diffusion Models
Weihao wu, Zhiwei Lin, Yixuan Zhou +6
Conversational speech synthesis (CSS) aims to synthesize both contextually appropriate and expressive speech, and considerable efforts have been made to enhance the understanding o…
Singing Voice Conversion with Accompaniment Using Self-Supervised Representation-Based Melody Features
Wei Chen, Binzhu Sha, Jing Yang +3
Melody preservation is crucial in singing voice conversion (SVC). However, in many scenarios, audio is often accompanied with background music (BGM), which can cause audio distorti…
The Codec Language Model-based Zero-Shot Spontaneous Style TTS System for CoVoC Challenge 2024
Shuoyi Zhou, Yixuan Zhou, Weiqin Li +6
This paper describes the zero-shot spontaneous style TTS system for the ISCSLP 2024 Conversational Voice Clone Challenge (CoVoC). We propose a LLaMA-based codec language model with…
RobustSVC: HuBERT-based Melody Extractor and Adversarial Learning for Robust Singing Voice Conversion
Wei Chen, Xintao Zhao, Jun Chen +3
Singing voice conversion (SVC) is hindered by noise sensitivity due to the use of non-robust methods for extracting pitch and energy during the inference. As clean signals are key…
MuCodec: Ultra Low-Bitrate Music Codec
Yaoxun Xu, Hangting Chen, Jianwei Yu +5
Music codecs are a vital aspect of audio codec research, and ultra low-bitrate compression holds significant importance for music transmission and generation. Due to the complexity…