Emotional Prosody Control for Speech Generation

Abstract

Machine-generated speech is characterized by its limited or unnatural emotional variation. Current text to speech systems generates speech with either a flat emotion, emotion selected from a predefined set, average variation learned from prosody sequences in training data or transferred from a source style. We propose a text to speech(TTS) system, where a user can choose the emotion of generated speech from a continuous and meaningful emotion space (Arousal-Valence space). The proposed TTS system can generate speech from the text in any speaker's style, with fine control of emotion. We show that the system works on emotion unseen during training and can scale to previously unseen speakers given his/her speech sample. Our work expands the horizon of the state-of-the-art FastSpeech2 backbone to a multi-speaker setting and gives it much-coveted continuous (and interpretable) affective control, without any observable degradation in the quality of the synthesized speech.

References in corpus (14)

• WaveNet: A Generative Model for Raw Audio
• VoxCeleb: a large-scale speaker identification dataset
• MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
• FastSpeech: Fast, Robust and Controllable Text to Speech
• Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
• FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
• Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
• Deep Voice: Real-time Neural Text-to-Speech
• Parallel WaveNet: Fast High-Fidelity Speech Synthesis
• Deep Voice 2: Multi-Speaker Neural Text-to-Speech
• CopyCat: Many-to-Many Fine-Grained Prosody Transfer for Neural Text-to-Speech
• Effective Use of Variational Embedding Capacity in Expressive End-to-End Speech Synthesis
• Semi-Supervised Generative Modeling for Controllable Speech Synthesis
• Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversion

Cited by in corpus (1)

• EmoSphere++: Emotion-Controllable Zero-Shot Text-to-Speech via Emotion-Adaptive Spherical Vector