Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversion

Abstract

End-to-end models for raw audio generation are a challenge, specially if they have to work with non-parallel data, which is a desirable setup in many situations. Voice conversion, in which a model has to impersonate a speaker in a recording, is one of those situations. In this paper, we propose Blow, a single-scale normalizing flow using hypernetwork conditioning to perform many-to-many voice conversion between raw audio. Blow is trained end-to-end, with non-parallel data, on a frame-by-frame basis using a single speaker identifier. We show that Blow compares favorably to existing flow-based architectures and other competitive baselines, obtaining equal or better performance in both objective and subjective evaluations. We further assess the impact of its main components with an ablation study, and quantify a number of properties such as the necessary amount of training data or the preference for source or target speakers.

arXiv commentIncludes appendix. Accepted for NeurIPS2019

References in corpus (14)

• WaveNet: A Generative Model for Raw Audio
• Neural Discrete Representation Learning
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• Deep Voice: Real-time Neural Text-to-Speech
• Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
• SampleRNN: An Unconditional End-to-End Neural Audio Generation Model
• Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design
• Adversarial Audio Synthesis
• ACVAE-VC: Non-parallel many-to-many voice conversion with auxiliary classifier variational autoencoder
• StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networks
• Emerging Convolutions for Generative Normalizing Flows
• A RAD approach to deep mixture models
• TzK Flow - Conditional Generative Model

Cited by in corpus (11)

• Adaptation Algorithms for Neural Network-Based Speech Recognition: An Overview
• WaveFlow: A Compact Flow-based Model for Raw Audio
• Towards Robust Neural Vocoding for Speech Generation: A Survey
• Review of end-to-end speech synthesis technology based on deep learning
• TGAVC: Improving Autoencoder Voice Conversion with Text-Guided and Adversarial Training
• VoiceGrad: Non-Parallel Any-to-Many Voice Conversion with Annealed Langevin Dynamics
• Many-to-Many Voice Transformer Network
• FastS2S-VC: Streaming Non-Autoregressive Sequence-to-Sequence Voice Conversion
• SNAC: Speaker-normalized affine coupling layer in flow-based architecture for zero-shot multi-speaker text-to-speech
• Nonparallel Voice Conversion with Augmented Classifier Star Generative Adversarial Networks
• Speak Like a Dog: Human to Non-human creature Voice Conversion