One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization

Abstract

Recently, voice conversion (VC) without parallel data has been successfully adapted to multi-target scenario in which a single model is trained to convert the input voice to many different speakers. However, such model suffers from the limitation that it can only convert the voice to the speakers in the training data, which narrows down the applicable scenario of VC. In this paper, we proposed a novel one-shot VC approach which is able to perform VC by only an example utterance from source and target speaker respectively, and the source and target speaker do not even need to be seen during training. This is achieved by disentangling speaker and content representations with instance normalization (IN). Objective and subjective evaluation shows that our model is able to generate the voice similar to target speaker. In addition to the performance measurement, we also demonstrate that this model is able to learn meaningful speaker representations without any supervision.

arXiv commentInterspeech 2019

References in corpus (14)

• Instance Normalization: The Missing Ingredient for Fast Stylization
• Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
• GANSynth: Adversarial Neural Audio Synthesis
• Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron
• Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks
• Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networks
• Tacotron: Towards End-to-End Speech Synthesis
• ACVAE-VC: Non-parallel many-to-many voice conversion with auxiliary classifier variational autoencoder
• StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networks
• Machine Speech Chain with One-shot Speaker Adaptation
• Voice Conversion Using Sequence-to-Sequence Learning of Context Posterior Probabilities
• WaveCycleGAN: Synthetic-to-natural speech waveform conversion using cycle-consistent adversarial networks
• Generative Adversarial Networks for Unpaired Voice Transformation on Impaired Speech
• Rhythm-Flexible Voice Conversion without Parallel Data Using Cycle-GAN over Phoneme Posteriorgram Sequences

Cited by in corpus (9)

• CopyCat: Many-to-Many Fine-Grained Prosody Transfer for Neural Text-to-Speech
• Fine-grained Style Modeling, Transfer and Prediction in Text-to-Speech Synthesis via Phone-Level Content-Style Disentanglement
• Emotional Voice Conversion: Theory, Databases and ESD
• ConVoice: Real-Time Zero-Shot Voice Style Transfer with Convolutional Network
• Unsupervised Audiovisual Synthesis via Exemplar Autoencoders
• MASS: Multi-task Anthropomorphic Speech Synthesis Framework
• FMFCC-A: A Challenging Mandarin Dataset for Synthetic Speech Detection
• Learning in your voice: Non-parallel voice conversion based on speaker consistency loss
• Accent and Speaker Disentanglement in Many-to-many Voice Conversion