Adversarially Trained Multi-Singer Sequence-To-Sequence Singing Synthesizer
arXiv:2006.10317
Abstract
This paper presents a high quality singing synthesizer that is able to model a voice with limited available recordings. Based on the sequence-to-sequence singing model, we design a multi-singer framework to leverage all the existing singing data of different singers. To attenuate the issue of musical score unbalance among singers, we incorporate an adversarial task of singer classification to make encoder output less singer dependent. Furthermore, we apply multiple random window discriminators (MRWDs) on the generated acoustic features to make the network be a GAN. Both objective and subjective evaluations indicate that the proposed synthesizer can generate higher quality singing voice than baseline (4.12 vs 3.53 in MOS). Especially, the articulation of high-pitched vowels is significantly enhanced.
Submitted to INTERSPEECH2020
References in corpus (9)
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Spectral Normalization for Generative Adversarial Networks
- MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generation
- High Fidelity Speech Synthesis with Adversarial Networks
- Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning
- Generative adversarial network-based glottal waveform model for statistical parametric speech synthesis
- Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloning
- Singing Voice Synthesis Using Deep Autoregressive Neural Networks for Acoustic Modeling
- PitchNet: Unsupervised Singing Voice Conversion with Pitch Adversarial Network
Cited by in corpus (3)
- A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directions
- Learn2Sing: Target Speaker Singing Voice Synthesis by learning from a Singing Teacher
- RefineGAN: Universally Generating Waveform Better than Ground Truth with Highly Accurate Pitch and Intensity Responses