Modeling Prosodic Phrasing with Multi-Task Learning in Tacotron-based TTS

Abstract

Tacotron-based end-to-end speech synthesis has shown remarkable voice quality. However, the rendering of prosody in the synthesized speech remains to be improved, especially for long sentences, where prosodic phrasing errors can occur frequently. In this paper, we extend the Tacotron-based speech synthesis framework to explicitly model the prosodic phrase breaks. We propose a multi-task learning scheme for Tacotron training, that optimizes the system to predict both Mel spectrum and phrase breaks. To our best knowledge, this is the first implementation of multi-task learning for Tacotron based TTS with a prosodic phrasing model. Experiments show that our proposed training scheme consistently improves the voice quality for both Chinese and Mongolian systems.

arXiv commentTo appear in IEEE Signal Processing Letters (SPL)

References in corpus (5)

• A Survey on Multi-Task Learning
• Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
• Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron
• Expressive Speech Synthesis via Modeling Expressions with Variational Autoencoder
• Feature reinforcement with word embedding and parsing information in neural TTS

Cited by in corpus (4)

• A Survey on Neural Speech Synthesis
• An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learning
• GraphSpeech: Syntax-Aware Graph Attention Network For Neural Speech Synthesis
• An investigation of phrase break prediction in an End-to-End TTS system