Integrating Uncertainty into Neural Network-based Speech Enhancement

Abstract

Supervised masking approaches in the time-frequency domain aim to employ deep neural networks to estimate a multiplicative mask to extract clean speech. This leads to a single estimate for each input without any guarantees or measures of reliability. In this paper, we study the benefits of modeling uncertainty in clean speech estimation. Prediction uncertainty is typically categorized into aleatoric uncertainty and epistemic uncertainty. The former refers to inherent randomness in data, while the latter describes uncertainty in the model parameters. In this work, we propose a framework to jointly model aleatoric and epistemic uncertainties in neural network-based speech enhancement. The proposed approach captures aleatoric uncertainty by estimating the statistical moments of the speech posterior distribution and explicitly incorporates the uncertainty estimate to further improve clean speech estimation. For epistemic uncertainty, we investigate two Bayesian deep learning approaches: Monte Carlo dropout and Deep ensembles to quantify the uncertainty of the neural network parameters. Our analyses show that the proposed framework promotes capturing practical and reliable uncertainty, while combining different sources of uncertainties yields more reliable predictive uncertainty estimates. Furthermore, we demonstrate the benefits of modeling uncertainty on speech enhancement performance by evaluating the framework on different datasets, exhibiting notable improvement over comparable models that fail to account for uncertainty.

arXiv commentAccepted version

References in corpus (22)

• Adam: A Method for Stochastic Optimization
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods
• Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
• Stochastic Gradient Hamiltonian Monte Carlo
• Deep Ensembles: A Loss Landscape Perspective
• Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding
• High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach
• Statistical Speech Enhancement Based on Probabilistic Integration of Variational Autoencoder and Non-Negative Matrix Factorization
• Audio-visual Speech Enhancement Using Conditional Variational Auto-Encoders
• Variational Autoencoder for Speech Enhancement with a Noise-Aware Encoder
• A variance modeling framework based on variational autoencoders for speech enhancement
• On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks
• SNR-Based Features and Diverse Training Data for Robust DNN-Based Speech Enhancement
• DNN-Based Speech Presence Probability Estimation for Multi-Frame Single-Microphone Speech Enhancement
• Guided Variational Autoencoder for Speech Enhancement With a Supervised Classifier
• Disentanglement Learning for Variational Autoencoders Applied to Audio-Visual Speech Enhancement
• Integrating Statistical Uncertainty into Neural Network-Based Speech Enhancement
• Variational Variance: Simple, Reliable, Calibrated Heteroscedastic Noise Variance Parameterization
• MetricGAN-U: Unsupervised speech enhancement/ dereverberation based only on noisy/ reverberated speech
• Noisy-target Training: A Training Strategy for DNN-based Speech Enhancement without Clean Speech