Voice Conversion with Conditional SampleRNN

Abstract

Here we present a novel approach to conditioning the SampleRNN generative model for voice conversion (VC). Conventional methods for VC modify the perceived speaker identity by converting between source and target acoustic features. Our approach focuses on preserving voice content and depends on the generative network to learn voice style. We first train a multi-speaker SampleRNN model conditioned on linguistic features, pitch contour, and speaker identity using a multi-speaker speech corpus. Voice-converted speech is generated using linguistic features and pitch contour extracted from the source speaker, and the target speaker identity. We demonstrate that our system is capable of many-to-many voice conversion without requiring parallel data, enabling broad applications. Subjective evaluation demonstrates that our approach outperforms conventional VC methods.

arXiv commentAccepted at Interspeech 2018, Hyderabad, India. This version matches the final version submitted to the conference

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• WaveNet: A Generative Model for Raw Audio
• Deep Voice: Real-time Neural Text-to-Speech
• SampleRNN: An Unconditional End-to-End Neural Audio Generation Model
• Deep Voice 2: Multi-Speaker Neural Text-to-Speech

Cited by in corpus (3)

• The Ethical Implications of Generative Audio Models: A Systematic Literature Review
• Adversarial Learning of Deepfakes in Accounting
• asya: Mindful verbal communication using deep learning