Sequence-to-sequence Singing Voice Synthesis with Perceptual Entropy Loss
arXiv:2010.12024
Abstract
The neural network (NN) based singing voice synthesis (SVS) systems require sufficient data to train well and are prone to over-fitting due to data scarcity. However, we often encounter data limitation problem in building SVS systems because of high data acquisition and annotation costs. In this work, we propose a Perceptual Entropy (PE) loss derived from a psycho-acoustic hearing model to regularize the network. With a one-hour open-source singing voice database, we explore the impact of the PE loss on various mainstream sequence-to-sequence models, including the RNN-based, transformer-based, and conformer-based models. Our experiments show that the PE loss can mitigate the over-fitting problem and significantly improve the synthesized singing quality reflected in objective and subjective evaluations.
arXiv commentAccepted by ICASSP2021
References in corpus (11)
- A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay
- Conformer: Convolution-augmented Transformer for Speech Recognition
- WGANSing: A Multi-Voice Singing Voice Synthesizer Based on the Wasserstein-GAN
- HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesis
- Singing voice synthesis based on convolutional neural networks
- XiaoiceSing: A High-Quality and Integrated Singing Voice Synthesis System
- ByteSing: A Chinese Singing Voice Synthesis System Using Duration Allocated Encoder-Decoder Acoustic Models and WaveRNN Vocoders
- Score and Lyrics-Free Singing Voice Generation
- DurIAN-SC: Duration Informed Attention Network based Singing Voice Conversion System
- Learning Singing From Speech
- Peking Opera Synthesis via Duration Informed Attention Network