Word-Level Style Control for Expressive, Non-attentive Speech Synthesis
arXiv:2111.10173 · doi:10.1007/978-3-030-87802-3_31
Abstract
This paper presents an expressive speech synthesis architecture for modeling and controlling the speaking style at a word level. It attempts to learn word-level stylistic and prosodic representations of the speech data, with the aid of two encoders. The first one models style by finding a combination of style tokens for each word given the acoustic features, and the second outputs a word-level sequence conditioned only on the phonetic information in order to disentangle it from the style information. The two encoder outputs are aligned and concatenated with the phoneme encoder outputs and then decoded with a Non-Attentive Tacotron model. An extra prior encoder is used to predict the style tokens autoregressively, in order for the model to be able to run without a reference utterance. We find that the resulting model gives both word-level and global control over the style, as well as prosody transfer capabilities.
arXiv commentProceedings of SPECOM 2021
References in corpus (11)
- Adam: A Method for Stochastic Optimization
- Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
- Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron
- DurIAN: Duration Informed Attention Network For Multimodal Synthesis
- Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling
- CopyCat: Many-to-Many Fine-Grained Prosody Transfer for Neural Text-to-Speech
- CHiVE: Varying Prosody in Speech Synthesis with a Linguistically Driven Dynamic Hierarchical Conditional Variational Network
- Uncovering Latent Style Factors for Expressive Speech Synthesis
- Hierarchical Generative Modeling for Controllable Speech Synthesis
- High Quality Streaming Speech Synthesis with Low, Sentence-Length-Independent Latency
- Generating diverse and natural text-to-speech samples using a quantized fine-grained VAE and auto-regressive prosody prior