Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks

Abstract

We propose a parallel-data-free voice-conversion (VC) method that can learn a mapping from source to target speech without relying on parallel data. The proposed method is general purpose, high quality, and parallel-data free and works without any extra data, modules, or alignment procedure. It also avoids over-smoothing, which occurs in many conventional statistical model-based VC methods. Our method, called CycleGAN-VC, uses a cycle-consistent adversarial network (CycleGAN) with gated convolutional neural networks (CNNs) and an identity-mapping loss. A CycleGAN learns forward and inverse mappings simultaneously using adversarial and cycle-consistency losses. This makes it possible to find an optimal pseudo pair from unpaired data. Furthermore, the adversarial loss contributes to reducing over-smoothing of the converted feature sequence. We configure a CycleGAN with gated CNNs and train it with an identity-mapping loss. This allows the mapping function to capture sequential and hierarchical structures while preserving linguistic information. We evaluated our method on a parallel-data-free VC task. An objective evaluation showed that the converted feature sequence was near natural in terms of global variance and modulation spectra. A subjective evaluation showed that the quality of the converted speech was comparable to that obtained with a Gaussian mixture model-based method under advantageous conditions with parallel and twice the amount of data.

References in corpus (1)

• Unsupervised Cross-Domain Image Generation

Cited by in corpus (30)

• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss
• Adversarial Speaker Verification
• WESPER: Zero-shot and Realtime Whisper to Normal Voice Conversion for Whisper-based Speech Interactions
• VSMask: Defending Against Voice Synthesis Attack via Real-Time Predictive Perturbation
• Global Rhythm Style Transfer Without Text Transcriptions
• Emotional Voice Conversion With Cycle-consistent Adversarial Network
• Joint training framework for text-to-speech and voice conversion using multi-source Tacotron and WaveNet
• CycleGAN-VC2: Improved CycleGAN-based Non-parallel Voice Conversion
• NoiseVC: Towards High Quality Zero-Shot Voice Conversion
• Non-parallel Voice Conversion System with WaveNet Vocoder and Collapsed Speech Suppression
• A Cycle-GAN Approach to Model Natural Perturbations in Speech for ASR Applications
• Hierarchical Sequence to Sequence Voice Conversion with Limited Data
• AGAIN-VC: A One-shot Voice Conversion using Activation Guidance and Adaptive Instance Normalization
• Attention-Guided Generative Adversarial Network for Whisper to Normal Speech Conversion
• VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversion
• Crossmodal Voice Conversion
• Building Multi lingual TTS using Cross Lingual Voice Conversion
• Expressive Voice Conversion: A Joint Framework for Speaker Identity and Emotional Style Transfer
• Many-to-Many Voice Conversion with Out-of-Dataset Speaker Support
• Transferring Source Style in Non-Parallel Voice Conversion
• asya: Mindful verbal communication using deep learning
• Investigation of F0 conditioning and Fully Convolutional Networks in Variational Autoencoder based Voice Conversion
• DarkGAN: Exploiting Knowledge Distillation for Comprehensible Audio Synthesis with GANs
• CycleGAN with Dual Adversarial Loss for Bone-Conducted Speech Enhancement
• MaskCycleGAN-VC: Learning Non-parallel Voice Conversion with Filling in Frames
• Improving robustness of one-shot voice conversion with deep discriminative speaker encoder
• A Deep-Bayesian Framework for Adaptive Speech Duration Modification
• Unsupervised Cross-Domain Speech-to-Speech Conversion with Time-Frequency Consistency
• Adversarially Trained Autoencoders for Parallel-Data-Free Voice Conversion