Feature reinforcement with word embedding and parsing information in neural TTS

Abstract

In this paper, we propose a feature reinforcement method under the sequence-to-sequence neural text-to-speech (TTS) synthesis framework. The proposed method utilizes the multiple input encoder to take three levels of text information, i.e., phoneme sequence, pre-trained word embedding, and grammatical structure of sentences from parser as the input feature for the neural TTS system. The added word and sentence level information can be viewed as the feature based pre-training strategy, which clearly enhances the model generalization ability. The proposed method not only improves the system robustness significantly but also improves the synthesized speech to near recording quality in our experiments for out-of-domain text.

References in corpus (1)

• WaveNet: A Generative Model for Raw Audio

Cited by in corpus (5)

• The Ethical Implications of Generative Audio Models: A Systematic Literature Review
• Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Models
• Modeling Prosodic Phrasing with Multi-Task Learning in Tacotron-based TTS
• Investigation of Japanese PnG BERT language model in text-to-speech synthesis for pitch accent language
• PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTS