Semi-blind source separation with multichannel variational autoencoder

Abstract

This paper proposes a multichannel source separation technique called the multichannel variational autoencoder (MVAE) method, which uses a conditional VAE (CVAE) to model and estimate the power spectrograms of the sources in a mixture. By training the CVAE using the spectrograms of training examples with source-class labels, we can use the trained decoder distribution as a universal generative model capable of generating spectrograms conditioned on a specified class label. By treating the latent space variables and the class label as the unknown parameters of this generative model, we can develop a convergence-guaranteed semi-blind source separation algorithm that consists of iteratively estimating the power spectrograms of the underlying sources as well as the separation matrices. In experimental evaluations, our MVAE produced better separation performance than a baseline method.

References in corpus (5)

• Auto-Encoding Variational Bayes
• Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks
• Statistical Speech Enhancement Based on Probabilistic Integration of Variational Autoencoder and Non-Negative Matrix Factorization
• The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods
• StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networks

Cited by in corpus (7)

• Dynamical Variational Autoencoders: A Comprehensive Review
• Audio Source Separation Using Variational Autoencoders and Weak Class Supervision
• A Deep Generative Model of Speech Complex Spectrograms
• Minimum-volume Multichannel Nonnegative matrix factorization for blind source separation
• Modeling the Repetition-based Recovering of Acoustic and Visual Sources with Dendritic Neurons
• Tomographic Auto-Encoder: Unsupervised Bayesian Recovery of Corrupted Data
• Deep Generative Variational Autoencoding for Replay Spoof Detection in Automatic Speaker Verification