Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset
arXiv:2010.14794
Abstract
Emotional voice conversion aims to transform emotional prosody in speech while preserving the linguistic content and speaker identity. Prior studies show that it is possible to disentangle emotional prosody using an encoder-decoder network conditioned on discrete representation, such as one-hot emotion labels. Such networks learn to remember a fixed set of emotional styles. In this paper, we propose a novel framework based on variational auto-encoding Wasserstein generative adversarial network (VAW-GAN), which makes use of a pre-trained speech emotion recognition (SER) model to transfer emotional style during training and at run-time inference. In this way, the network is able to transfer both seen and unseen emotional style to a new utterance. We show that the proposed framework achieves remarkable performance by consistently outperforming the baseline framework. This paper also marks the release of an emotional speech dataset (ESD) for voice conversion, which has multiple speakers and languages.
arXiv commentAccepted by ICASSP 2021
References in corpus (6)
- Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
- Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networks
- Nonparallel Emotional Speech Conversion
- Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends
- Expressive TTS Training with Frame and Style Reconstruction Loss
- Interactive Text-to-Speech System via Joint Style Analysis
Cited by in corpus (5)
- Expressive TTS Training with Frame and Style Reconstruction Loss
- GraphSpeech: Syntax-Aware Graph Attention Network For Neural Speech Synthesis
- VAW-GAN for Singing Voice Conversion with Non-parallel Training Data
- An Improved StarGAN for Emotional Voice Conversion: Enhancing Voice Quality and Data Augmentation
- Spectrum and Prosody Conversion for Cross-lingual Voice Conversion with CycleGAN