Semi-supervised multichannel speech enhancement with variational autoencoders and non-negative matrix factorization

Abstract

In this paper we address speaker-independent multichannel speech enhancement in unknown noisy environments. Our work is based on a well-established multichannel local Gaussian modeling framework. We propose to use a neural network for modeling the speech spectro-temporal content. The parameters of this supervised model are learned using the framework of variational autoencoders. The noisy recording environment is supposed to be unknown, so the noise spectro-temporal modeling remains unsupervised and is based on non-negative matrix factorization (NMF). We develop a Monte Carlo expectation-maximization algorithm and we experimentally show that the proposed approach outperforms its NMF-based counterpart, where speech is modeled using supervised NMF.

arXiv comment5 pages, 2 figures, audio examples and code available online at https://team.inria.fr/perception/icassp-2019-mvae/

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• Jensen's Operator Inequality
• Statistical Speech Enhancement Based on Probabilistic Integration of Variational Autoencoder and Non-Negative Matrix Factorization
• Speech enhancement with variational autoencoders and alpha-stable distributions

Cited by in corpus (12)

• A Survey of Sound Source Localization with Deep Learning Methods
• Dynamical Variational Autoencoders: A Comprehensive Review
• Audio-visual Speech Enhancement Using Conditional Variational Auto-Encoders
• Variational Autoencoder for Speech Enhancement with a Noise-Aware Encoder
• Speech enhancement with variational autoencoders and alpha-stable distributions
• Autoencoders for music sound modeling: a comparison of linear, shallow, deep, recurrent and variational models
• Joint NN-Supported Multichannel Reduction of Acoustic Echo, Reverberation and Noise
• Learning and controlling the source-filter representation of speech with a variational autoencoder
• Learning robust speech representation with an articulatory-regularized variational autoencoder
• Fast Multichannel Source Separation Based on Jointly Diagonalizable Spatial Covariance Matrices
• FastMVAE2: On improving and accelerating the fast variational autoencoder-based source separation algorithm for determined mixtures
• A Benchmark of Dynamical Variational Autoencoders applied to Speech Spectrogram Modeling