Masked Autoregressive Flow for Density Estimation

Abstract

Autoregressive models are among the best performing neural density estimators. We describe an approach for increasing the flexibility of an autoregressive model, based on modelling the random numbers that the model uses internally when generating data. By constructing a stack of autoregressive models, each modelling the random numbers of the next model in the stack, we obtain a type of normalizing flow suitable for density estimation, which we call Masked Autoregressive Flow. This type of flow is closely related to Inverse Autoregressive Flow and is a generalization of Real NVP. Masked Autoregressive Flow achieves state-of-the-art performance in a range of general-purpose density estimation tasks.

section 4.3 is corrected since the previous version

References in corpus (4)

• WaveNet: A Generative Model for Raw Audio
• NICE: Non-linear Independent Components Estimation
• MADE: Masked Autoencoder for Distribution Estimation
• High-Dimensional Probability Estimation with Deep Density Models

Cited by in corpus (51)

• The frontier of simulation-based inference
• Mining gold from implicit models to improve likelihood-free inference
• Roundtrip: A Deep Generative Neural Density Estimator
• Nonparametric Density Estimation for High-Dimensional Data - Algorithms and Applications
• Mining for Dark Matter Substructure: Inferring subhalo population properties from strong lenses with machine learning
• Rare and Different: Anomaly Scores from a combination of likelihood and out-of-distribution models to detect new physics at the LHC
• Quasi Anomalous Knowledge: Searching for new physics with embedded knowledge
• Fixing a Broken ELBO
• Dynamical mass inference of galaxy clusters with neural flows
• Monge-Ampère Flow for Generative Modeling
• Conditional Density Estimation with Bayesian Normalising Flows
• Differentiable Strong Lensing: Uniting Gravity and Neural Nets through Differentiable Probabilistic Programming
• IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression
• On the representation and learning of monotone triangular transport maps
• A Tutorial on Deep Latent Variable Models of Natural Language
• Simulation-based inference of dynamical galaxy cluster masses with 3D convolutional neural networks
• CaloFlow II: Even Faster and Still Accurate Generation of Calorimeter Showers with Normalizing Flows
• Flexible distribution-free conditional predictive bands using density estimators
• Nested sampling with any prior you like
• Learning the temporal evolution of multivariate densities via normalizing flows
• Effective LHC measurements with matrix elements and machine learning
• Multimap targeted free energy estimation
• Transformation Autoregressive Networks
• Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
• Charting Galactic Accelerations: When and How to Extract a Unique Potential from the Distribution Function
• NFAD: Fixing anomaly detection using normalizing flows
• Temporal Normalizing Flows
• Coupling the reduced-order model and the generative model for an importance sampling estimator
• ChartPointFlow for Topology-Aware 3D Point Cloud Generation
• Comparison of Affine and Rational Quadratic Spline Coupling and Autoregressive Flows through Robust Statistical Tests
• Normalizing flows for random fields in cosmology
• Improvements to Inference Compilation for Probabilistic Programming in Large-Scale Scientific Simulators
• Modeling the Gaia Color-Magnitude Diagram with Bayesian Neural Flows to Constrain Distance Estimates
• Improving Sequential Latent Variable Models with Autoregressive Flows
• Sparse Flows: Pruning Continuous-depth Models
• A novel stellar spectrum denoising method based on deep Bayesian modeling
• VAE-KRnet and its applications to variational Bayes
• Featurized Density Ratio Estimation
• An In-depth Summary of Recent Artificial Intelligence Applications in Drug Design
• Inference and De-Noising of Non-Gaussian Particle Distribution Functions: A Generative Modeling Approach
• Manifold-preserved GANs
• Towards Robust Active Feature Acquisition
• Augmented KRnet for density estimation and approximation
• Efficient Multimodal Sampling via Tempered Distribution Flow
• Resampling Base Distributions of Normalizing Flows
• TzK Flow - Conditional Generative Model
• ADAVI: Automatic Dual Amortized Variational Inference Applied To Pyramidal Bayesian Models
• Meta-analysis of dichotomous and ordinal tests without a gold standard
• Improving Compositionality of Neural Networks by Decoding Representations to Inputs
• Physics Informed Convex Artificial Neural Networks (PICANNs) for Optimal Transport based Density Estimation
• Hierarchical Reinforcement Learning with Optimal Level Synchronization Based on Flow-Based Deep Generative Model