Prosody Transfer in Neural Text to Speech Using Global Pitch and Loudness Features

Abstract

This paper presents a simple yet effective method to achieve prosody transfer from a reference speech signal to synthesized speech. The main idea is to incorporate well-known acoustic correlates of prosody such as pitch and loudness contours of the reference speech into a modern neural text-to-speech (TTS) synthesizer such as Tacotron2 (TC2). More specifically, a small set of acoustic features are extracted from reference audio and then used to condition a TC2 synthesizer. The trained model is evaluated using subjective listening tests and a novel objective evaluation of prosody transfer is proposed. Listening tests show that the synthesized speech is rated as highly natural and that prosody is successfully transferred from the reference speech signal to the synthesized signal.

arXiv comment5 pages, in review for conference publication

References in corpus (4)

• Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
• Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron
• Tacotron: Towards End-to-End Speech Synthesis
• CHiVE: Varying Prosody in Speech Synthesis with a Linguistically Driven Dynamic Hierarchical Conditional Variational Network

Cited by in corpus (8)

• A Survey on Neural Speech Synthesis
• Review of end-to-end speech synthesis technology based on deep learning
• Controllable speech synthesis by learning discrete phoneme-level prosodic representations
• Prosodic Clustering for Phoneme-level Prosody Control in End-to-End Speech Synthesis
• MASS: Multi-task Anthropomorphic Speech Synthesis Framework
• Improved Prosodic Clustering for Multispeaker and Speaker-independent Phoneme-level Prosody Control
• Pitchtron: Towards audiobook generation from ordinary people's voices
• Information Sieve: Content Leakage Reduction in End-to-End Prosody For Expressive Speech Synthesis