Uncovering Latent Style Factors for Expressive Speech Synthesis
arXiv:1711.00520
Abstract
Prosodic modeling is a core problem in speech synthesis. The key challenge is producing desirable prosody from textual input containing only phonetic information. In this preliminary study, we introduce the concept of "style tokens" in Tacotron, a recently proposed end-to-end neural speech synthesis model. Using style tokens, we aim to extract independent prosodic styles from training data. We show that without annotation data or an explicit supervision signal, our approach can automatically learn a variety of prosodic variations in a purely data-driven way. Importantly, each style token corresponds to a fixed style factor regardless of the given text sequence. As a result, we can control the prosodic style of synthetic speech in a somewhat predictable and globally consistent way.
arXiv commentSubmitted to NIPS ML4Audio workshop and ICASSP
References in corpus (1)
Cited by in corpus (16)
- Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron
- Deep Encoder-Decoder Models for Unsupervised Learning of Controllable Speech Synthesis
- Expressive Speech Synthesis via Modeling Expressions with Variational Autoencoder
- MuSE-SVS: Multi-Singer Emotional Singing Voice Synthesizer that Controls Emotional Intensity
- Interactive Text-to-Speech System via Joint Style Analysis
- Word-Level Style Control for Expressive, Non-attentive Speech Synthesis
- From Speaker Verification to Multispeaker Speech Synthesis, Deep Transfer with Feedback Constraint
- GraphTTS: graph-to-sequence modelling in neural text-to-speech
- LaughNet: synthesizing laughter utterances from waveform silhouettes and a single laughter example
- GraphPB: Graphical Representations of Prosody Boundary in Speech Synthesis
- Forward-Backward Decoding for Regularizing End-to-End TTS
- A Deep-Bayesian Framework for Adaptive Speech Duration Modification
- Controllable Emotion Transfer For End-to-End Speech Synthesis
- Emotional speech synthesis with rich and granularized control
- Modeling of Rakugo Speech and Its Limitations: Toward Speech Synthesis That Entertains Audiences
- GANtron: Emotional Speech Synthesis with Generative Adversarial Networks