Mining gold from implicit models to improve likelihood-free inference

Abstract

Simulators often provide the best description of real-world phenomena. However, they also lead to challenging inverse problems because the density they implicitly define is often intractable. We present a new suite of simulation-based inference techniques that go beyond the traditional Approximate Bayesian Computation approach, which struggles in a high-dimensional setting, and extend methods that use surrogate models based on neural networks. We show that additional information, such as the joint likelihood ratio and the joint score, can often be extracted from simulators and used to augment the training data for these surrogate models. Finally, we demonstrate that these new techniques are more sample efficient and provide higher-fidelity inference than traditional methods.

Code available at https://github.com/johannbrehmer/simulator-mining-example . v2: Fixed typos. v3: Expanded discussion, added Lotka-Volterra example. v4: Improved clarity

References in corpus (40)

• The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations
• Generative Adversarial Networks
• WaveNet: A Generative Model for Raw Audio
• Conditional Image Generation with PixelCNN Decoders
• Pixel Recurrent Neural Networks
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• The frontier of simulation-based inference
• Density estimation using Real NVP
• Pyro: Deep Universal Probabilistic Programming
• MADE: Masked Autoencoder for Distribution Estimation
• Parameterized Machine Learning for High-Energy Physics
• Masked Autoregressive Flow for Density Estimation
• Learning in Implicit Generative Models
• Mining gold from implicit models to improve likelihood-free inference
• Constraining Effective Field Theories with Machine Learning
• Massive optimal data compression and density estimation for scalable, likelihood-free inference in cosmology
• A Guide to Constraining Effective Field Theories with Machine Learning
• Fast $\epsilon$-free Inference of Simulation Models with Bayesian Conditional Density Estimation
• Learning Disentangled Representations with Semi-Supervised Deep Generative Models
• Approximating Likelihood Ratios with Calibrated Discriminative Classifiers
• FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
• Nuisance hardened data compression for fast likelihood-free inference
• Automatic physical inference with information maximising neural networks
• Generalized massive optimal data compression
• Flexible statistical inference for mechanistic models of neural dynamics
• Hierarchical Implicit Models and Likelihood-Free Variational Inference
• Deep Probabilistic Programming
• Mining for Dark Matter Substructure: Inferring subhalo population properties from strong lenses with machine learning
• Likelihood-free inference with emulator networks
• Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows
• Mad-Maximized Higgs Pair Analyses
• Inference Networks for Sequential Monte Carlo in Graphical Models
• Maximum Significance at the LHC and Higgs Decays to Muons
• Inference Compilation and Universal Probabilistic Programming
• Likelihood-free MCMC with Amortized Approximate Ratio Estimators
• Neural Autoregressive Distribution Estimation
• Adversarial Variational Optimization of Non-Differentiable Simulators
• MadMax, or Where Boosted Significances Come From
• Dynamic Likelihood-free Inference via Ratio Estimation (DIRE)

Cited by in corpus (84)

• Machine learning and the physical sciences
• The frontier of simulation-based inference
• Fast likelihood-free cosmology with neural density estimators and active learning
• Learning New Physics from a Machine
• OmniFold: A Method to Simultaneously Unfold All Observables
• Mining gold from implicit models to improve likelihood-free inference
• Simulation Assisted Likelihood-free Anomaly Detection
• Adversarially-trained autoencoders for robust unsupervised new physics searches
• Guiding New Physics Searches with Unsupervised Learning
• Nuisance hardened data compression for fast likelihood-free inference
• Machine Learning and LHC Event Generation
• Neural Networks for Full Phase-space Reweighting and Parameter Tuning
• Mining for Dark Matter Substructure: Inferring subhalo population properties from strong lenses with machine learning
• Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows
• A Neural Resampler for Monte Carlo Reweighting with Preserved Uncertainties
• Model calibration using ESEm v1.0.0 -- an open, scalable Earth System Emulator
• A guide for deploying Deep Learning in LHC searches: How to achieve optimality and account for uncertainty
• Uncertainty Aware Learning for High Energy Physics
• Towards the ultimate differential SMEFT analysis
• INFERNO: Inference-Aware Neural Optimisation
• Presenting Unbinned Differential Cross Section Results
• DCTRGAN: Improving the Precision of Generative Models with Reweighting
• A neural simulation-based inference approach for characterizing the Galactic Center $\gamma$-ray excess
• Machine learning the Higgs boson-top quark CP phase
• E Pluribus Unum Ex Machina: Learning from Many Collider Events at Once
• The LHC Olympics 2020: A Community Challenge for Anomaly Detection in High Energy Physics
• Accelerated Bayesian SED Modeling using Amortized Neural Posterior Estimation
• Constraining CP-violation in the Higgs-top-quark interaction using machine-learning-based inference
• Flows for simultaneous manifold learning and density estimation
• Parametrized classifiers for optimal EFT sensitivity
• A Cautionary Tale of Decorrelating Theory Uncertainties
• Towards constraining warm dark matter with stellar streams through neural simulation-based inference
• Likelihood-free MCMC with Amortized Approximate Ratio Estimators
• Minimax Confidence Intervals for the Sliced Wasserstein Distance
• Unbinned multivariate observables for global SMEFT analyses from machine learning
• Effective LHC measurements with matrix elements and machine learning
• Benchmarking Simulation-Based Inference
• Neural Approximate Sufficient Statistics for Implicit Models
• Tree boosting for learning EFT parameters
• ELSA -- Enhanced latent spaces for improved collider simulations
• Likelihood-free inference with an improved cross-entropy estimator
• Matrix Element Regression with Deep Neural Networks -- breaking the CPU barrier
• Back to the Formula -- LHC Edition
• Evidence Networks: simple losses for fast, amortized, neural Bayesian model comparison
• Parameter Estimation using Neural Networks in the Presence of Detector Effects
• Measurement of off-shell Higgs boson production in the $H^{\ast }\to ZZ\to 4\ell$ decay channel using a neural simulation-based inference technique in 13 TeV $pp$ collisions with the ATLAS detector
• Learning Multivariate New Physics
• Reframing Jet Physics with New Computational Methods
• Differentiable Matrix Elements with MadJax
• On the impact of selected modern deep-learning techniques to the performance and celerity of classification models in an experimental high-energy physics use case
• Bagged filters for partially observed interacting systems
• Neural Likelihood Surfaces for Spatial Processes with Computationally Intensive or Intractable Likelihoods
• Simulation-based inference of single-molecule experiments
• Parameter Inference from Event Ensembles and the Top-Quark Mass
• Simulation-based inference in the search for CP violation in leptonic WH production
• Simulation-Based Inference Benchmark for Weak Lensing Cosmology
• Spectropolarimetric Inversion in Four Dimensions with Deep Learning (SPIn4D): I. Overview, Magnetohydrodynamic Modeling, and Stokes Profile Synthesis
• Comparison of Affine and Rational Quadratic Spline Coupling and Autoregressive Flows through Robust Statistical Tests
• SIDE-real: Supernova Ia Dust Extinction with truncated marginal neural ratio estimation applied to real data
• Advancing Tools for Simulation-Based Inference
• Neural Density Estimation and Likelihood-free Inference
• Interpretable Uncertainty Quantification in AI for HEP
• Full-waveform earthquake source inversion using simulation-based inference
• Dealing with Nuisance Parameters using Machine Learning in High Energy Physics: a Review
• Neural Conditional Reweighting
• OASIS: Optimal Analysis-Specific Importance Sampling for event generation
• A rotation-equivariant graph neural network for learning hadronic SMEFT effects
• Refinable modeling for unbinned SMEFT analyses
• Confidence Sets and Hypothesis Testing in a Likelihood-Free Inference Setting
• Unbinned inclusive cross-section measurements with machine-learned systematic uncertainties
• LHC EFT WG Note: SMEFT predictions, event reweighting, and simulation
• Top-philic Machine Learning
• Constraints on standard model effective field theory for a Higgs boson produced in association with W or Z bosons in the H $\to \mathrm{b}\bar{\mathrm{b}}$ decay channel in proton-proton collisions at $\sqrt{s}$ = 13 TeV
• Analytic auto-differentiable $\Lambda$CDM cosmography
• Learning to discover: expressive Gaussian mixture models for multi-dimensional simulation and parameter inference in the physical sciences
• BitHEP -- The Limits of Low-Precision ML in HEP
• Equivariant neural networks for robust $\text{CP}$ observables
• Designing Observables for Measurements with Deep Learning
• Amortised Learning by Wake-Sleep
• Group equivariant neural posterior estimation
• Learning New Physics from an Imperfect Machine
• Searching for HWW Anomalous Couplings with Simulation-Based Inference
• LSST Strong Lensing Systems Dark Matter Sensitivity Analysis with Neural Ratio Estimators
• Using holistic event information in the trigger