Deep Forest: Neural Network reconstruction of the Lyman-alpha forest

Abstract

We explore the use of Deep Learning to infer physical quantities from the observable transmitted flux in the Lyman-alpha forest. We train a Neural Network using redshift z=3 outputs from cosmological hydrodynamic simulations and mock datasets constructed from them. We evaluate how well the trained network is able to reconstruct the optical depth for Lyman-alpha forest absorption from noisy and often saturated transmitted flux data. The Neural Network outperforms an alternative reconstruction method involving log inversion and spline interpolation by approximately a factor of 2 in the optical depth root mean square error. We find no significant dependence in the improvement on input data signal to noise, although the gain is greatest in high optical depth regions. The Lyman-alpha forest optical depth studied here serves as a simple, one dimensional, example but the use of Deep Learning and simulations to approach the inverse problem in cosmology could be extended to other physical quantities and higher dimensional data.

arXiv comment10 pages, 7 figures, submitted to MNRAS. Code and data used at https://github.com/lhuangCMU/deep-learning-intergalactic-medium Changes: 11 pages, 7 figures. Further described how we chose our architecture, why the NN has difficulty predicting high values of optical depth, added more references, added additional panels to figs 3-6, and corrected fig 1 and mean optical depth value

References in corpus (48)

• Adam: A Method for Stochastic Optimization
• Decoupled Weight Decay Regularization
• The cosmological simulation code GADGET-2
• A systematic study of the class imbalance problem in convolutional neural networks
• Cosmological SPH simulations: A hybrid multi-phase model for star formation
• Radiative Transfer in a Clumpy Universe: II. The Utraviolet Extragalactic Background
• The Completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: Cosmological Implications from two Decades of Spectroscopic Surveys at the Apache Point observatory
• Equation of State of the Photoionized Intergalactic Medium
• Cosmological implications of baryon acoustic oscillation (BAO) measurements
• Damped Lyman alpha Systems
• The Lyman Alpha Forest in the Spectra of QSOs
• Gravitational Collapse of Small-Scale Structure as the Origin of the Lyman Alpha Forest
• The Lyman-alpha Forest in the Cold Dark Matter Model
• Deep Learning for Real-time Gravitational Wave Detection and Parameter Estimation: Results with Advanced LIGO Data
• Cold flows and the first quasars
• A Multi-Species Model for Hydrogen and Helium Absorbers in Lyman-Alpha Forest Clouds
• Generative Adversarial Networks recover features in astrophysical images of galaxies beyond the deconvolution limit
• Damped Lyman Alpha Systems in Galaxy Formation Simulations
• Learning to Predict the Cosmological Structure Formation
• The Strong Gravitational Lens Finding Challenge
• Cosmological constraints with deep learning from KiDS-450 weak lensing maps
• Optimising Automatic Morphological Classification of Galaxies with Machine Learning and Deep Learning using Dark Energy Survey Imaging
• Automatic physical inference with information maximising neural networks
• XQ-100: A legacy survey of one hundred 3.5 < z < 4.5 quasars observed with VLT/XSHOOTER
• Weak lensing cosmology with convolutional neural networks on noisy data
• AI-assisted super-resolution cosmological simulations
• Inversion of the Lyman-$\alpha$ forest: 3D investigation of the intergalactic medium
• The BOSS Lyman-alpha Forest Sample from SDSS Data Release 9
• Deep Learning of Quasar Spectra to Discover and Characterize Damped Lya Systems
• DeepCMB: Lensing Reconstruction of the Cosmic Microwave Background with Deep Neural Networks
• Super-resolution emulator of cosmological simulations using deep physical models
• A first step towards a direct inversion of the Lyman forest in QSO spectra
• Machine learning cosmological structure formation
• LATIS: The Ly$\alpha$ Tomography IMACS Survey
• Intensity mapping with SDSS/BOSS Lyman-alpha emission, quasars and their Lyman-alpha forest
• DASH: Deep Learning for the Automated Spectral Classification of Supernovae and their Hosts
• TARDIS Paper I: A Constrained Reconstruction Approach to Modeling the z~2.5 Cosmic Web Probed by Lyman-alpha Forest Tomography
• The Lyman-alpha Forest as a Cosmological Tool
• The Lyman-alpha forest of a Lyman Break Galaxy: VLT Spectra of MS1512-cB58 at z=2.724
• Nonparametric 3D map of the IGM using the Lyman-alpha forest
• HIGAN: Cosmic Neutral Hydrogen with Generative Adversarial Networks
• Revealing the Local Cosmic Web from Galaxies by Deep Learning
• Artificial intelligence and quasar absorption system modelling; application to fundamental constants at high redshift
• Trend Filtering -- I. A Modern Statistical Tool for Time-Domain Astronomy and Astronomical Spectroscopy
• Fully probabilistic quasar continua predictions near Lyman-α with conditional neural spline flows
• Recovering dark-matter clustering from galaxies with Gaussianization
• Trend Filtering -- II. Denoising Astronomical Signals with Varying Degrees of Smoothness
• An Optimized Ly$\alpha$ Forest Inversion Tool Based on a Quantitative Comparison of Existing Reconstruction Methods

Cited by in corpus (8)

• Extracting cosmological parameters from N-body simulations using machine learning techniques
• Mapping Lyman-alpha forest three-dimensional large scale structure in real and redshift space
• A Novel Estimator for the Equation of State of the IGM by Ly$\alpha$ Forest Tomography
• Ly$\alpha$NNA: A Deep Learning Field-level Inference Machine for the Lyman-$\alpha$ Forest
• Deep Forest: Neural Network reconstruction of intergalactic medium temperature
• Reconstructing Large-scale Temperature Profiles around $z\sim 6$ Quasars
• Neural network emulator to constrain the high-$z$ IGM thermal state from Lyman-$\alpha$ forest flux auto-correlation function
• Efficient neutral-IGM inference from noisy 21-cm forest spectra with latent-space U-Net encoding and XGBoost