9 papers
Controllable Text-to-Speech Synthesis with Masked-Autoencoded Style-Rich Representation
Yongqi Wang, Chunlei Zhang, Hangting Chen +2
Controllable TTS models with natural language prompts often lack the ability for fine-grained control and face a scarcity of high-quality data. We propose a two-stage style-control…
TechSinger: Technique Controllable Multilingual Singing Voice Synthesis via Flow Matching
Wenxiang Guo, Yu Zhang, Changhao Pan +6
Singing voice synthesis has made remarkable progress in generating natural and high-quality voices. However, existing methods rarely provide precise control over vocal techniques s…
MoMu-Diffusion: On Learning Long-Term Motion-Music Synchronization and Correspondence
Fuming You, Minghui Fang, Li Tang +3
Motion-to-music and music-to-motion have been studied separately, each attracting substantial research interest within their respective domains. The interaction between human motio…
Accompanied Singing Voice Synthesis with Fully Text-controlled Melody
Ruiqi Li, Zhiqing Hong, Yongqi Wang +4
Text-to-song (TTSong) is a music generation task that synthesizes accompanied singing voices. Current TTSong methods, inherited from singing voice synthesis (SVS), require melody-r…
Self-Supervised Singing Voice Pre-Training towards Speech-to-Singing Conversion
Ruiqi Li, Rongjie Huang, Yongqi Wang +2
Speech-to-singing voice conversion (STS) task always suffers from data scarcity, because it requires paired speech and singing data. Compounding this issue are the challenges of co…
Robust Singing Voice Transcription Serves Synthesis
Ruiqi Li, Yu Zhang, Yongqi Wang +3
Note-level Automatic Singing Voice Transcription (AST) converts singing recordings into note sequences, facilitating the automatic annotation of singing datasets for Singing Voice…
Frieren: Efficient Video-to-Audio Generation Network with Rectified Flow Matching
Yongqi Wang, Wenxiang Guo, Rongjie Huang +5
Video-to-audio (V2A) generation aims to synthesize content-matching audio from silent video, and it remains challenging to build V2A models with high generation quality, efficiency…
Text-to-Song: Towards Controllable Music Generation Incorporating Vocals and Accompaniment
Zhiqing Hong, Rongjie Huang, Xize Cheng +5
A song is a combination of singing voice and accompaniment. However, existing works focus on singing voice synthesis and music generation independently. Little attention was paid t…
Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt
Yongqi Wang, Ruofan Hu, Rongjie Huang +6
Recent singing-voice-synthesis (SVS) methods have achieved remarkable audio quality and naturalness, yet they lack the capability to control the style attributes of the synthesized…