Parameters Estimation for the Cosmic Microwave Background with Bayesian Neural Networks

Abstract

In this paper, we present the first study that compares different models of Bayesian Neural Networks (BNNs) to predict the posterior distribution of the cosmological parameters directly from the Cosmic Microwave Background temperature and polarization maps. We focus our analysis on four different methods to sample the weights of the network during training: Dropout, DropConnect, Reparameterization Trick (RT), and Flipout. We find out that Flipout outperforms all other methods regardless of the architecture used, and provides tighter constraints for the cosmological parameters. Moreover we compare with MCMC posterior analysis obtaining comparable error correlation among parameters, with BNNs being orders of magnitude faster in inference, although less accurate. Thanks to the speed of the inference process with BNNs, the posterior distribution, outcome of the neural network, can be used as the initial proposal for the Markov Chain. We show that this combined approach increases the acceptance rate in the Metropolis-Hasting algorithm and accelerates the convergence of the MCMC, while reaching the same final accuracy. In the second part of the paper, we present a guide to the training and calibration of a successful multi-channel BNN for the CMB temperature and polarization map. We show how tuning the regularization parameter for the standard deviation of the approximate posterior on the weights in Flipout and RT we can produce unbiased and reliable uncertainty estimates, i.e., the regularizer acts like a hyperparameter analogous to the dropout rate in Dropout. Finally, we show how polarization, when combined with the temperature in a unique multi-channel tensor fed to a single BNN, helps to break degeneracies among parameters and provides stringent constraints.

24 pages with 19 figures

References in corpus (61)

• Auto-Encoding Variational Bayes
• Planck 2015 results. XIII. Cosmological parameters
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• The Cosmic Linear Anisotropy Solving System (CLASS) II: Approximation schemes
• On Calibration of Modern Neural Networks
• Bayes in the sky: Bayesian inference and model selection in cosmology
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks
• Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
• Efficient sampling of fast and slow cosmological parameters
• Neural network models and deep learning - a primer for biologists
• CMB power spectrum parameter degeneracies in the era of precision cosmology
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Cosmic Confusion: Degeneracies among Cosmological Parameters Derived from Measurements of Microwave Background Anisotropies
• Precision big bang nucleosynthesis with improved Helium-4 predictions
• Why does deep and cheap learning work so well?
• Regularizing Neural Networks by Penalizing Confident Output Distributions
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
• Accurate Uncertainties for Deep Learning Using Calibrated Regression
• Fast and reliable MCMC for cosmological parameter estimation
• Fast Automated Analysis of Strong Gravitational Lenses with Convolutional Neural Networks
• Learning to Predict the Cosmological Structure Formation
• CMU DeepLens: Deep Learning For Automatic Image-based Galaxy-Galaxy Strong Lens Finding
• Finding Strong Gravitational Lenses in the Kilo Degree Survey with Convolutional Neural Networks
• Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches
• DeepSphere: Efficient spherical Convolutional Neural Network with HEALPix sampling for cosmological applications
• Cosmological constraints with deep learning from KiDS-450 weak lensing maps
• Uncertainties in Parameters Estimated with Neural Networks: Application to Strong Gravitational Lensing
• Non-Gaussian information from weak lensing data via deep learning
• Deep learning from 21-cm tomography of the Cosmic Dawn and Reionization
• Weak lensing cosmology with convolutional neural networks on noisy data
• Cosmological constraints from noisy convergence maps through deep learning
• CMB Polarization Experiments
• Hamilton's approach in cosmological inflation with an exponential potential and its observational constraints
• DeepCMB: Lensing Reconstruction of the Cosmic Microwave Background with Deep Neural Networks
• An Ensemble of Bayesian Neural Networks for Exoplanetary Atmospheric Retrieval
• Neutrino masses and beyond-$\Lambda$CDM cosmology with LSST and future CMB experiments
• Uncertainty Estimations by Softplus normalization in Bayesian Convolutional Neural Networks with Variational Inference
• Mocking the Weak Lensing universe: the LensTools python computing package
• Effective and Efficient Dropout for Deep Convolutional Neural Networks
• Convolutional Neural Networks on the HEALPix sphere: a pixel-based algorithm and its application to CMB data analysis
• Loss-Calibrated Approximate Inference in Bayesian Neural Networks
• CosmicNet I: Physics-driven implementation of neural networks within Boltzmann-Einstein solvers
• Euclid-era cosmology for everyone: Neural net assisted MCMC sampling for the joint 3x2 likelihood
• Cosmological model discrimination with Deep Learning
• Analyzing interferometric observations of strong gravitational lenses with recurrent and convolutional neural networks
• Improved supervised learning methods for EoR parameters reconstruction
• A Convolutional Neural Network For Cosmic String Detection in CMB Temperature Maps
• A Neural-Network based estimator to search for primordial non-Gaussianity in Planck CMB maps
• Probing the Early Universe with the CMB Scalar, Vector and Tensor Bispectrum
• Calibrating Uncertainties in Object Localization Task
• Dark Energy Constraints in light of Pantheon SNe Ia, BAO, Cosmic Chronometers and CMB Polarization and Lensing Data
• Robustly representing uncertainty in deep neural networks through sampling
• Reduced bispectrum seeded by helical primordial magnetic fields
• Foreground removal from WMAP 5yr temperature maps using an MLP neural network
• Why CMB physics?
• CMB Polarization as complementary information to anisotropies
• Confirmation of the detection of B-modes in the Planck polarization maps
• Reliable Uncertainties for Bayesian Neural Networks using Alpha-divergences
• Pico: Parameters for the Impatient Cosmologist
• Excess B-modes extracted from the Planck polarization maps

Cited by in corpus (30)

• Deeply Uncertain: Comparing Methods of Uncertainty Quantification in Deep Learning Algorithms
• Simulation-Based Inference of Reionization Parameters From 3D Tomographic 21 cm Lightcone Images
• The GCE in a New Light: Disentangling the $\gamma$-ray Sky with Bayesian Graph Convolutional Neural Networks
• Hierarchical Inference With Bayesian Neural Networks: An Application to Strong Gravitational Lensing
• $\mathtt{bajes}$: Bayesian inference of multimessenger astrophysical data, methods and application to gravitational-waves
• Cosmology Intertwined: A Review of the Particle Physics, Astrophysics, and Cosmology Associated with the Cosmological Tensions and Anomalies
• Neural network reconstructions for the Hubble parameter, growth rate and distance modulus
• Variational Inference as an alternative to MCMC for parameter estimation and model selection
• Spectroscopic Studies of Type Ia Supernovae Using LSTM Neural Networks
• Photometric redshift estimates using Bayesian neural networks in the CSST survey
• DeepSZ: Identification of Sunyaev-Zel'dovich Galaxy Clusters using Deep Learning
• LeMoN: Lens Modelling with Neural networks -- I. Automated modelling of strong gravitational lenses with Bayesian Neural Networks
• Reconstructing Cosmic Polarization Rotation with ResUNet-CMB
• Dark energy reconstruction analysis with artificial neural networks: Application on simulated Supernova Ia data from Rubin Observatory
• Forecasting VIX using Bayesian Deep Learning
• Test of Artificial Neural Networks in Likelihood-free Cosmological Constraints: A Comparison of IMNN and DAE
• Constraining cosmological parameters from N-body simulations with Variational Bayesian Neural Networks
• Deep Learning and genetic algorithms for cosmological Bayesian inference speed-up
• Deep Bayesian segmentation for colon polyps: Well-calibrated predictions in medical imaging
• Bayesian deep learning for cosmic volumes with modified gravity
• Estimating Photometric Redshifts for Galaxies from the DESI Legacy Imaging Surveys with Bayesian Neural Networks Trained by DESI EDR
• Accurately Estimating Redshifts from CSST Slitless Spectroscopic Survey using Deep Learning
• Measuring the Substructure Mass Power Spectrum of 23 SLACS Strong Galaxy-Galaxy Lenses with Convolutional Neural Networks
• Accelerating MCMC algorithms through Bayesian Deep Networks
• Reliable Uncertainties for Bayesian Neural Networks using Alpha-divergences
• Unsupervised Domain Adaptation for Constraining Star Formation Histories
• Understanding the predication mechanism of deep learning through error propagation among parameters in strong lensing case
• Deep inference of simulated strong lenses in ground-based surveys
• Parameters Estimation from the 21 cm signal using Variational Inference
• Bayesian classification of astronomical spectra with class uncertainties