Score and Lyrics-Free Singing Voice Generation

Abstract

Generative models for singing voice have been mostly concerned with the task of ``singing voice synthesis,'' i.e., to produce singing voice waveforms given musical scores and text lyrics. In this work, we explore a novel yet challenging alternative: singing voice generation without pre-assigned scores and lyrics, in both training and inference time. In particular, we outline three such generation schemes, and propose a pipeline to tackle these new tasks. Moreover, we implement such models using generative adversarial networks and evaluate them both objectively and subjectively.

arXiv commentAccepted by International Conference on Computational Creativity (ICCC) 2020

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• Conditional Generative Adversarial Nets
• WaveNet: A Generative Model for Raw Audio
• Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generation
• GANSynth: Adversarial Neural Audio Synthesis
• MelNet: A Generative Model for Audio in the Frequency Domain
• High Fidelity Speech Synthesis with Adversarial Networks
• Adversarial Audio Synthesis
• Music Transformer
• Dense Prediction on Sequences with Time-Dilated Convolutions for Speech Recognition
• Chord Generation from Symbolic Melody Using BLSTM Networks
• madmom: a new Python Audio and Music Signal Processing Library
• Singing voice synthesis based on convolutional neural networks
• Dilated Convolution with Dilated GRU for Music Source Separation

Cited by in corpus (10)

• A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directions
• Unconditional Audio Generation with Generative Adversarial Networks and Cycle Regularization
• VQVC+: One-Shot Voice Conversion by Vector Quantization and U-Net architecture
• Addressing the confounds of accompaniments in singer identification
• Speech-to-Singing Conversion in an Encoder-Decoder Framework
• Adversarial attacks on audio source separation
• Speech-to-Singing Conversion based on Boundary Equilibrium GAN
• A Benchmarking Initiative for Audio-Domain Music Generation Using the Freesound Loop Dataset
• MusicTM-Dataset for Joint Representation Learning among Sheet Music, Lyrics, and Musical Audio
• Sequence-to-sequence Singing Voice Synthesis with Perceptual Entropy Loss