Hierarchical Emotion Prediction and Control in Text-to-Speech Synthesis
arXiv:2405.09171 · doi:10.1109/ICASSP48485.2024.10445996
Abstract
It remains a challenge to effectively control the emotion rendering in text-to-speech (TTS) synthesis. Prior studies have primarily focused on learning a global prosodic representation at the utterance level, which strongly correlates with linguistic prosody. Our goal is to construct a hierarchical emotion distribution (ED) that effectively encapsulates intensity variations of emotions at various levels of granularity, encompassing phonemes, words, and utterances. During TTS training, the hierarchical ED is extracted from the ground-truth audio and guides the predictor to establish a connection between emotional and linguistic prosody. At run-time inference, the TTS model generates emotional speech and, at the same time, provides quantitative control of emotion over the speech constituents. Both objective and subjective evaluations validate the effectiveness of the proposed framework in terms of emotion prediction and control.
This is accepted to IEEE ICASSP 2024
References in corpus (7)
- Adam: A Method for Stochastic Optimization
- HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
- Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
- FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
- A Survey on Neural Speech Synthesis
- Emotion Intensity and its Control for Emotional Voice Conversion
- An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era