Neural Autoregressive Distribution Estimation
arXiv:1605.02226
Abstract
We present Neural Autoregressive Distribution Estimation (NADE) models, which are neural network architectures applied to the problem of unsupervised distribution and density estimation. They leverage the probability product rule and a weight sharing scheme inspired from restricted Boltzmann machines, to yield an estimator that is both tractable and has good generalization performance. We discuss how they achieve competitive performance in modeling both binary and real-valued observations. We also present how deep NADE models can be trained to be agnostic to the ordering of input dimensions used by the autoregressive product rule decomposition. Finally, we also show how to exploit the topological structure of pixels in images using a deep convolutional architecture for NADE.
References in corpus (12)
- Auto-Encoding Variational Bayes
- Pixel Recurrent Neural Networks
- Generative Moment Matching Networks
- MADE: Masked Autoencoder for Distribution Estimation
- Deep AutoRegressive Networks
- RNADE: The real-valued neural autoregressive density-estimator
- Generative Image Modeling Using Spatial LSTMs
- Learning Feature Hierarchies with Centered Deep Boltzmann Machines
- A Deep and Autoregressive Approach for Topic Modeling of Multimodal Data
- Minimum Probability Flow Learning
- Deep Mixtures of Factor Analysers
- Iterative Neural Autoregressive Distribution Estimator (NADE-k)
Cited by in corpus (10)
- Mining gold from implicit models to improve likelihood-free inference
- Mining for Dark Matter Substructure: Inferring subhalo population properties from strong lenses with machine learning
- Inference of the optical depth to reionization from low multipole temperature and polarisation Planck data
- Dynamical mass inference of galaxy clusters with neural flows
- Effective LHC measurements with matrix elements and machine learning
- jVMC: Versatile and performant variational Monte Carlo leveraging automated differentiation and GPU acceleration
- Impact of conditional modelling for a universal autoregressive quantum state
- Locally Masked Convolution for Autoregressive Models
- A novel stellar spectrum denoising method based on deep Bayesian modeling
- Limitations of Autoregressive Models and Their Alternatives