Semi-Supervised Generative Modeling for Controllable Speech Synthesis
arXiv:1910.01709
Abstract
We present a novel generative model that combines state-of-the-art neural text-to-speech (TTS) with semi-supervised probabilistic latent variable models. By providing partial supervision to some of the latent variables, we are able to force them to take on consistent and interpretable purposes, which previously hasn't been possible with purely unsupervised TTS models. We demonstrate that our model is able to reliably discover and control important but rarely labelled attributes of speech, such as affect and speaking rate, with as little as 1% (30 minutes) supervision. Even at such low supervision levels we do not observe a degradation of synthesis quality compared to a state-of-the-art baseline. Audio samples are available on the web.
References in corpus (13)
- Semi-Supervised Learning with Deep Generative Models
- Disentangling by Factorising
- FastSpeech: Fast, Robust and Controllable Text to Speech
- Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
- Disentangling factors of variation in deep representations using adversarial training
- Tacotron: Towards End-to-End Speech Synthesis
- MelNet: A Generative Model for Audio in the Frequency Domain
- Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning
- DIVA: Domain Invariant Variational Autoencoders
- Efficient Neural Audio Synthesis
- CHiVE: Varying Prosody in Speech Synthesis with a Linguistically Driven Dynamic Hierarchical Conditional Variational Network
- Effective Use of Variational Embedding Capacity in Expressive End-to-End Speech Synthesis
- Expressive Speech Synthesis via Modeling Expressions with Variational Autoencoder
Cited by in corpus (8)
- A Survey on Neural Speech Synthesis
- Emotional Prosody Control for Speech Generation
- Music FaderNets: Controllable Music Generation Based On High-Level Features via Low-Level Feature Modelling
- Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesis
- Controllable speech synthesis by learning discrete phoneme-level prosodic representations
- Prosodic Clustering for Phoneme-level Prosody Control in End-to-End Speech Synthesis
- An Improved Semi-Supervised VAE for Learning Disentangled Representations
- WaveTTS: Tacotron-based TTS with Joint Time-Frequency Domain Loss