Voice Conversion from Non-parallel Corpora Using Variational Auto-encoder

Abstract

We propose a flexible framework for spectral conversion (SC) that facilitates training with unaligned corpora. Many SC frameworks require parallel corpora, phonetic alignments, or explicit frame-wise correspondence for learning conversion functions or for synthesizing a target spectrum with the aid of alignments. However, these requirements gravely limit the scope of practical applications of SC due to scarcity or even unavailability of parallel corpora. We propose an SC framework based on variational auto-encoder which enables us to exploit non-parallel corpora. The framework comprises an encoder that learns speaker-independent phonetic representations and a decoder that learns to reconstruct the designated speaker. It removes the requirement of parallel corpora or phonetic alignments to train a spectral conversion system. We report objective and subjective evaluations to validate our proposed method and compare it to SC methods that have access to aligned corpora.

References in corpus (1)

• Auto-Encoding Variational Bayes

Cited by in corpus (10)

• AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss
• Deep Encoder-Decoder Models for Unsupervised Learning of Controllable Speech Synthesis
• StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networks
• Expressive Speech Synthesis via Modeling Expressions with Variational Autoencoder
• Voice Conversion Using Sequence-to-Sequence Learning of Context Posterior Probabilities
• Investigation of Using Disentangled and Interpretable Representations for One-shot Cross-lingual Voice Conversion
• AttS2S-VC: Sequence-to-Sequence Voice Conversion with Attention and Context Preservation Mechanisms
• Towards Fine-Grained Prosody Control for Voice Conversion
• Error Reduction Network for DBLSTM-based Voice Conversion
• Adversarially learning disentangled speech representations for robust multi-factor voice conversion