Neural Approximate Sufficient Statistics for Implicit Models

Abstract

We consider the fundamental problem of how to automatically construct summary statistics for implicit generative models where the evaluation of the likelihood function is intractable, but sampling data from the model is possible. The idea is to frame the task of constructing sufficient statistics as learning mutual information maximizing representations of the data with the help of deep neural networks. The infomax learning procedure does not need to estimate any density or density ratio. We apply our approach to both traditional approximate Bayesian computation and recent neural likelihood methods, boosting their performance on a range of tasks.

arXiv commentICLR2021 spotlight

References in corpus (23)

• Adam: A Method for Stochastic Optimization
• A Brief Introduction to PYTHIA 8.1
• Learning deep representations by mutual information estimation and maximization
• Pixel Recurrent Neural Networks
• Estimating divergence functionals and the likelihood ratio by convex risk minimization
• Adaptive approximate Bayesian computation
• A Comparative Review of Dimension Reduction Methods in Approximate Bayesian Computation
• A survey of dimensionality reduction techniques
• Mining gold from implicit models to improve likelihood-free inference
• Massive optimal data compression and density estimation for scalable, likelihood-free inference in cosmology
• Fast $\epsilon$-free Inference of Simulation Models with Bayesian Conditional Density Estimation
• Approximating Likelihood Ratios with Calibrated Discriminative Classifiers
• Likelihood-Free Cosmological Inference with Type Ia Supernovae: Approximate Bayesian Computation for a Complete Treatment of Uncertainty
• Flexible statistical inference for mechanistic models of neural dynamics
• Automatic Posterior Transformation for Likelihood-Free Inference
• High-Dimensional Probability Estimation with Deep Density Models
• Approximate Bayesian Image Interpretation using Generative Probabilistic Graphics Programs
• A survey of dimensionality reduction techniques based on random projection
• Neural Spline Flows
• Benchmarking Simulation-Based Inference
• Dynamic Likelihood-free Inference via Ratio Estimation (DIRE)
• Partially Exchangeable Networks and Architectures for Learning Summary Statistics in Approximate Bayesian Computation
• Adaptive Gaussian Copula ABC

Cited by in corpus (5)

• Black-box Bayesian inference for economic agent-based models
• Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
• Cosmological Parameter Estimation and Inference using Deep Summaries
• A Generalizable Model-and-Data Driven Approach for Open-Set RFF Authentication
• Towards Data-Conditional Simulation for ABC Inference in Stochastic Differential Equations