MASS: Multi-task Anthropomorphic Speech Synthesis Framework

Abstract

Text-to-Speech (TTS) synthesis plays an important role in human-computer interaction. Currently, most TTS technologies focus on the naturalness of speech, namely,making the speeches sound like humans. However, the key tasks of the expression of emotion and the speaker identity are ignored, which limits the application scenarios of TTS synthesis technology. To make the synthesized speech more realistic and expand the application scenarios, we propose a multi-task anthropomorphic speech synthesis framework (MASS), which can synthesize speeches from text with specified emotion and speaker identity. The MASS framework consists of a base TTS module and two novel voice conversion modules: the emotional voice conversion module and the speaker voice conversion module. We propose deep emotion voice conversion model (DEVC) and deep speaker voice conversion model (DSVC) based on convolution residual networks. It solves the problem of feature loss during voice conversion. The model trainings are independent of parallel datasets, and are capable of many-to-many voice conversion. In the emotional voice conversion, speaker voice conversion experiments, as well as the multi-task speech synthesis experiments, experimental results show DEVC and DSVC convert speech effectively. The quantitative and qualitative evaluation results of multi-task speech synthesis experiments show MASS can effectively synthesis speech with specified text, emotion and speaker identity.

References in corpus (19)

• Instance Normalization: The Missing Ingredient for Fast Stylization
• Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
• Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
• Deep Voice: Real-time Neural Text-to-Speech
• Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron
• Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks
• Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networks
• Emotional End-to-End Neural Speech Synthesizer
• Deep Encoder-Decoder Models for Unsupervised Learning of Controllable Speech Synthesis
• Hierarchical Generative Modeling for Controllable Speech Synthesis
• Multi-reference Tacotron by Intercross Training for Style Disentangling,Transfer and Control in Speech Synthesis
• Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretraining
• One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization
• StarGAN-VC2: Rethinking Conditional Methods for StarGAN-Based Voice Conversion
• Transforming Spectrum and Prosody for Emotional Voice Conversion with Non-Parallel Training Data
• Prosody Transfer in Neural Text to Speech Using Global Pitch and Loudness Features
• Converting Anyone's Emotion: Towards Speaker-Independent Emotional Voice Conversion
• Joint training framework for text-to-speech and voice conversion using multi-source Tacotron and WaveNet
• Multi-Reference Neural TTS Stylization with Adversarial Cycle Consistency