Rhythm-Flexible Voice Conversion without Parallel Data Using Cycle-GAN over Phoneme Posteriorgram Sequences
arXiv:1808.03113
Abstract
Speaking rate refers to the average number of phonemes within some unit time, while the rhythmic patterns refer to duration distributions for realizations of different phonemes within different phonetic structures. Both are key components of prosody in speech, which is different for different speakers. Models like cycle-consistent adversarial network (Cycle-GAN) and variational auto-encoder (VAE) have been successfully applied to voice conversion tasks without parallel data. However, due to the neural network architectures and feature vectors chosen for these approaches, the length of the predicted utterance has to be fixed to that of the input utterance, which limits the flexibility in mimicking the speaking rates and rhythmic patterns for the target speaker. On the other hand, sequence-to-sequence learning model was used to remove the above length constraint, but parallel training data are needed. In this paper, we propose an approach utilizing sequence-to-sequence model trained with unsupervised Cycle-GAN to perform the transformation between the phoneme posteriorgram sequences for different speakers. In this way, the length constraint mentioned above is removed to offer rhythm-flexible voice conversion without requiring parallel data. Preliminary evaluation on two datasets showed very encouraging results.
arXiv comment8 pages, 6 figures, Submitted to SLT 2018
References in corpus (9)
- Sequence to Sequence Learning with Neural Networks
- Conditional Generative Adversarial Nets
- WaveNet: A Generative Model for Raw Audio
- Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
- Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks
- Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networks
- Tacotron: Towards End-to-End Speech Synthesis
- StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networks
- Voice Conversion Using Sequence-to-Sequence Learning of Context Posterior Probabilities