A Benchmark of Dynamical Variational Autoencoders applied to Speech Spectrogram Modeling

Abstract

The Variational Autoencoder (VAE) is a powerful deep generative model that is now extensively used to represent high-dimensional complex data via a low-dimensional latent space learned in an unsupervised manner. In the original VAE model, input data vectors are processed independently. In recent years, a series of papers have presented different extensions of the VAE to process sequential data, that not only model the latent space, but also model the temporal dependencies within a sequence of data vectors and corresponding latent vectors, relying on recurrent neural networks. We recently performed a comprehensive review of those models and unified them into a general class called Dynamical Variational Autoencoders (DVAEs). In the present paper, we present the results of an experimental benchmark comparing six of those DVAE models on the speech analysis-resynthesis task, as an illustration of the high potential of DVAEs for speech modeling.

arXiv commentAccepted to Interspeech 2021. arXiv admin note: text overlap with arXiv:2008.12595

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Auto-Encoding Variational Bayes
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• A Recurrent Latent Variable Model for Sequential Data
• Learning Stochastic Recurrent Networks
• Variational Recurrent Auto-Encoders
• Sequential Neural Models with Stochastic Layers
• Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data
• Statistical Speech Enhancement Based on Probabilistic Integration of Variational Autoencoder and Non-Negative Matrix Factorization
• Deep Kalman Filters
• A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
• Disentangled Sequential Autoencoder
• Semi-supervised multichannel speech enhancement with variational autoencoders and non-negative matrix factorization
• Speech enhancement with variational autoencoders and alpha-stable distributions
• A Deep Generative Model of Speech Complex Spectrograms