Bayesian neural networks and dimensionality reduction
arXiv:2008.08044
Abstract
In conducting non-linear dimensionality reduction and feature learning, it is common to suppose that the data lie near a lower-dimensional manifold. A class of model-based approaches for such problems includes latent variables in an unknown non-linear regression function; this includes Gaussian process latent variable models and variational auto-encoders (VAEs) as special cases. VAEs are artificial neural networks (ANNs) that employ approximations to make computation tractable; however, current implementations lack adequate uncertainty quantification in estimating the parameters, predictive densities, and lower-dimensional subspace, and can be unstable and lack interpretability in practice. We attempt to solve these problems by deploying Markov chain Monte Carlo sampling algorithms (MCMC) for Bayesian inference in ANN models with latent variables. We address issues of identifiability by imposing constraints on the ANN parameters as well as by using anchor points. This is demonstrated on simulated and real data examples. We find that current MCMC sampling schemes face fundamental challenges in neural networks involving latent variables, motivating new research directions.
29 pages, 13 figures
References in corpus (13)
- A Recurrent Latent Variable Model for Sequential Data
- Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
- Stochastic Gradient Hamiltonian Monte Carlo
- Linear Dimensionality Reduction: Survey, Insights, and Generalizations
- Functional Variational Bayesian Neural Networks
- Structured Disentangled Representations
- A Hybrid Convolutional Variational Autoencoder for Text Generation
- Reversible Jump MCMC Simulated Annealing for Neural Networks
- Understanding Priors in Bayesian Neural Networks at the Unit Level
- Overpruning in Variational Bayesian Neural Networks
- Unbiased Implicit Variational Inference
- Semi-Separable Hamiltonian Monte Carlo for Inference in Bayesian Hierarchical Models
- Hierarchical Importance Weighted Autoencoders