Improved Prosodic Clustering for Multispeaker and Speaker-independent Phoneme-level Prosody Control
arXiv:2111.10168 · doi:10.1007/978-3-030-87802-3_11
Abstract
This paper presents a method for phoneme-level prosody control of F0 and duration on a multispeaker text-to-speech setup, which is based on prosodic clustering. An autoregressive attention-based model is used, incorporating multispeaker architecture modules in parallel to a prosody encoder. Several improvements over the basic single-speaker method are proposed that increase the prosodic control range and coverage. More specifically we employ data augmentation, F0 normalization, balanced clustering for duration, and speaker-independent prosodic clustering. These modifications enable fine-grained phoneme-level prosody control for all speakers contained in the training set, while maintaining the speaker identity. The model is also fine-tuned to unseen speakers with limited amounts of data and it is shown to maintain its prosody control capabilities, verifying that the speaker-independent prosodic clustering is effective. Experimental results verify that the model maintains high output speech quality and that the proposed method allows efficient prosody control within each speaker's range despite the variability that a multispeaker setting introduces.
Proceedings of SPECOM 2021
References in corpus (11)
- CopyCat: Many-to-Many Fine-Grained Prosody Transfer for Neural Text-to-Speech
- CHiVE: Varying Prosody in Speech Synthesis with a Linguistically Driven Dynamic Hierarchical Conditional Variational Network
- High Quality Streaming Speech Synthesis with Low, Sentence-Length-Independent Latency
- Fine-grained Style Modeling, Transfer and Prediction in Text-to-Speech Synthesis via Phone-Level Content-Style Disentanglement
- Expressive Neural Voice Cloning
- Generating diverse and natural text-to-speech samples using a quantized fine-grained VAE and auto-regressive prosody prior
- Prosodic Clustering for Phoneme-level Prosody Control in End-to-End Speech Synthesis
- Voice Conversion by Cascading Automatic Speech Recognition and Text-to-Speech Synthesis with Prosody Transfer
- Few Shot Adaptive Normalization Driven Multi-Speaker Speech Synthesis
- Adversarially learning disentangled speech representations for robust multi-factor voice conversion
- Can Speaker Augmentation Improve Multi-Speaker End-to-End TTS?