Speech-to-Singing Conversion based on Boundary Equilibrium GAN

Abstract

This paper investigates the use of generative adversarial network (GAN)-based models for converting the spectrogram of a speech signal into that of a singing one, without reference to the phoneme sequence underlying the speech. This is achieved by viewing speech-to-singing conversion as a style transfer problem. Specifically, given a speech input, and optionally the F0 contour of the target singing, the proposed model generates as the output a singing signal with a progressive-growing encoder/decoder architecture and boundary equilibrium GAN loss functions. Our quantitative and qualitative analysis show that the proposed model generates singing voices with much higher naturalness than an existing non adversarially-trained baseline. For reproducibility, the code will be publicly available at a GitHub repository upon paper publication.

Accepted for publication at INTERSPEECH 2020

References in corpus (12)

• Instance Normalization: The Missing Ingredient for Fast Stylization
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
• Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions
• MelNet: A Generative Model for Audio in the Frequency Domain
• Jukebox: A Generative Model for Music
• A Universal Music Translation Network
• Unconditional Audio Generation with Generative Adversarial Networks and Cycle Regularization
• Singing voice synthesis based on convolutional neural networks
• ByteSing: A Chinese Singing Voice Synthesis System Using Duration Allocated Encoder-Decoder Acoustic Models and WaveRNN Vocoders
• Score and Lyrics-Free Singing Voice Generation
• Singing Style Transfer Using Cycle-Consistent Boundary Equilibrium Generative Adversarial Networks