Automatic physical inference with information maximising neural networks

Abstract

Compressing large data sets to a manageable number of summaries that are informative about the underlying parameters vastly simplifies both frequentist and Bayesian inference. When only simulations are available, these summaries are typically chosen heuristically, so they may inadvertently miss important information. We introduce a simulation-based machine learning technique that trains artificial neural networks to find non-linear functionals of data that maximise Fisher information: information maximising neural networks (IMNNs). In test cases where the posterior can be derived exactly, likelihood-free inference based on automatically derived IMNN summaries produces nearly exact posteriors, showing that these summaries are good approximations to sufficient statistics. In a series of numerical examples of increasing complexity and astrophysical relevance we show that IMNNs are robustly capable of automatically finding optimal, non-linear summaries of the data even in cases where linear compression fails: inferring the variance of Gaussian signal in the presence of noise; inferring cosmological parameters from mock simulations of the Lyman-α forest in quasar spectra; and inferring frequency-domain parameters from LISA-like detections of gravitational waveforms. In this final case, the IMNN summary outperforms linear data compression by avoiding the introduction of spurious likelihood maxima. We anticipate that the automatic physical inference method described in this paper will be essential to obtain both accurate and precise cosmological parameter estimates from complex and large astronomical data sets, including those from LSST and Euclid.

21 pages, 12 figures, code at https://github.com/tomcharnock/information_maximiser, data at https://doi.org/10.5281/zenodo.1175196

References in corpus (6)

• The star formation histories of galaxies in the Sloan Digital Sky Survey
• Constraining the thick disc formation scenario of the Milky Way
• Massive optimal data compression and density estimation for scalable, likelihood-free inference in cosmology
• Generalized massive optimal data compression
• Massive data compression for parameter-dependent covariance matrices
• An investigation into the Multiple Optimised Parameter Estimation and Data compression algorithm

Cited by in corpus (78)

• Machine learning and the physical sciences
• The Quijote simulations
• Fast likelihood-free cosmology with neural density estimators and active learning
• Mining gold from implicit models to improve likelihood-free inference
• Cosmological constraints with deep learning from KiDS-450 weak lensing maps
• A Guide to Constraining Effective Field Theories with Machine Learning
• Likelihood-free inference with neural compression of DES SV weak lensing map statistics
• Nuisance hardened data compression for fast likelihood-free inference
• Bayesian optimisation for likelihood-free cosmological inference
• Cosmology From CMB Lensing and Delensed EE Power Spectra Using 2019-2020 SPT-3G Polarization Data
• The GIGANTES dataset: precision cosmology from voids in the machine learning era
• Fewer Mocks and Less Noise: Reducing the Dimensionality of Cosmological Observables with Subspace Projections
• Super-resolution emulator of cosmological simulations using deep physical models
• deep21: a Deep Learning Method for 21cm Foreground Removal
• Fast and Credible Likelihood-Free Cosmology with Truncated Marginal Neural Ratio Estimation
• Uncertainty Aware Learning for High Energy Physics
• A Full $w$CDM Analysis of KiDS-1000 Weak Lensing Maps using Deep Learning
• Translation and Rotation Equivariant Normalizing Flow (TRENF) for Optimal Cosmological Analysis
• INFERNO: Inference-Aware Neural Optimisation
• Field Level Neural Network Emulator for Cosmological N-body Simulations
• Lossless, Scalable Implicit Likelihood Inference for Cosmological Fields
• The Cosmic Graph: Optimal Information Extraction from Large-Scale Structure using Catalogues
• Dynamical mass inference of galaxy clusters with neural flows
• Cosmic Shear: Inference from Forward Models
• Kernel-Based Emulator for the 3D Matter Power Spectrum from CLASS
• CosmoGridV1: a simulated $w$CDM theory prediction for map-level cosmological inference
• Likelihood Non-Gaussianity in Large-Scale Structure Analyses
• A Cautionary Tale of Decorrelating Theory Uncertainties
• $\mathrm{S}\mathrm{IM}\mathrm{BIG}$: Mock Challenge for a Forward Modeling Approach to Galaxy Clustering
• Simulation-based inference of dynamical galaxy cluster masses with 3D convolutional neural networks
• Robust Simulation-Based Inference in Cosmology with Bayesian Neural Networks
• Likelihood-free Cosmological Constraints with Artificial Neural Networks: An Application on Hubble Parameters and SNe Ia
• Parameter Inference for Weak Lensing using Gaussian Processes and MOPED
• HIFlow: Generating Diverse HI Maps and Inferring Cosmology while Marginalizing over Astrophysics using Normalizing Flows
• Reducing the dependence of the neural network function to systematic uncertainties in the input space
• Benchmarking Simulation-Based Inference
• 21st Century Statistical and Computational Challenges in Astrophysics
• Effective LHC measurements with matrix elements and machine learning
• Unbiased likelihood-free inference of the Hubble constant from light standard sirens
• How informative are summaries of the cosmic 21-cm signal?
• A comparative study of cosmological constraints from weak lensing using Convolutional Neural Networks
• Data Compression and Inference in Cosmology with Self-Supervised Machine Learning
• Optimal statistical inference in the presence of systematic uncertainties using neural network optimization based on binned Poisson likelihoods with nuisance parameters
• Extreme data compression while searching for new physics
• Cosmology with Persistent Homology: a Fisher Forecast
• Cosmological Parameter Estimation and Inference using Deep Summaries
• Machine Learning and Cosmology
• Quantification of high dimensional non-Gaussianities and its implication to Fisher analysis in cosmology
• A Framework for Obtaining Accurate Posteriors of Strong Gravitational Lensing Parameters with Flexible Priors and Implicit Likelihoods using Density Estimation
• GEOMAX: beyond linear compression for 3pt galaxy clustering statistics
• Deep Forest: Neural Network reconstruction of the Lyman-alpha forest
• Bias and Priors in Machine Learning Calibrations for High Energy Physics
• Simulation-Based Inference Benchmark for Weak Lensing Cosmology
• Test of Artificial Neural Networks in Likelihood-free Cosmological Constraints: A Comparison of IMNN and DAE
• SIDE-real: Supernova Ia Dust Extinction with truncated marginal neural ratio estimation applied to real data
• Neural Density Estimation and Likelihood-free Inference
• A tomographic spherical mass map emulator of the KiDS-1000 survey using conditional generative adversarial networks
• Catalog-free modeling of galaxy types in deep images: Massive dimensional reduction with neural networks
• Almanac: MCMC-based signal extraction of power spectra and maps on the sphere
• Full-waveform earthquake source inversion using simulation-based inference
• Dealing with Nuisance Parameters using Machine Learning in High Energy Physics: a Review
• Simulation-based Anomaly Detection for Multileptons at the LHC
• Simulation-based population inference of LISA's Galactic binaries: Bypassing the global fit
• Cosmic topology. Part IVa. Classification of manifolds using machine learning: a case study with small toroidal universes
• Lightweight starshade position sensing with convolutional neural networks and simulation-based inference
• Simulation-based inference has its own Dodelson-Schneider effect (but it knows that it does)
• CLAP. I. Resolving miscalibration for deep learning-based galaxy photometric redshift estimation
• Hybrid summary statistics: neural weak lensing inference beyond the power spectrum
• Analytic auto-differentiable $\Lambda$CDM cosmography
• Detecting Modeling Bias with Continuous Time Flow Models on Weak Lensing Maps
• Insights on Galaxy Evolution from Interpretable Sparse Feature Networks
• JERALD: high-fidelity dark matter, stellar mass and neutral hydrogen maps from fast N-body simulations
• Parameter estimation from Ly$\alpha$ forest in Fourier space using Information Maximising Neural Network
• Deep inference of simulated strong lenses in ground-based surveys
• Forecast constraints on null tests of the $\Lambda$CDM model with SPHEREx
• Diagnosing Systematic Effects Using the Inferred Initial Power Spectrum
• Skew-spectra: a generalization to spin-$s$
• Beyond the Power Spectrum: A New Framework for Non-Stationary Fields with Applications to Light-Cone Effects in Line Intensity Mapping