StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesis

Abstract

Style transfer for out-of-domain (OOD) singing voice synthesis (SVS) focuses on generating high-quality singing voices with unseen styles (such as timbre, emotion, pronunciation, and articulation skills) derived from reference singing voice samples. However, the endeavor to model the intricate nuances of singing voice styles is an arduous task, as singing voices possess a remarkable degree of expressiveness. Moreover, existing SVS methods encounter a decline in the quality of synthesized singing voices in OOD scenarios, as they rest upon the assumption that the target vocal attributes are discernible during the training phase. To overcome these challenges, we propose StyleSinger, the first singing voice synthesis model for zero-shot style transfer of out-of-domain reference singing voice samples. StyleSinger incorporates two critical approaches for enhanced effectiveness: 1) the Residual Style Adaptor (RSA) which employs a residual quantization module to capture diverse style characteristics in singing voices, and 2) the Uncertainty Modeling Layer Normalization (UMLN) to perturb the style attributes within the content representation during the training phase and thus improve the model generalization. Our extensive evaluations in zero-shot style transfer undeniably establish that StyleSinger outperforms baseline models in both audio quality and similarity to the reference singing voice samples. Access to singing voice samples can be found at https://aaronz345.github.io/StyleSingerDemo/.

Accepted by AAAI 2024

References in corpus (8)

• AdaSpeech: Adaptive Text to Speech for Custom Voice
• Uncertainty Modeling for Out-of-Distribution Generalization
• BigVGAN: A Universal Neural Vocoder with Large-Scale Training
• Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation
• DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism
• ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech
• VISinger: Variational Inference with Adversarial Learning for End-to-End Singing Voice Synthesis
• MuSE-SVS: Multi-Singer Emotional Singing Voice Synthesizer that Controls Emotional Intensity