Deep Learning of Quasar Spectra to Discover and Characterize Damped Lya Systems

Abstract

We have designed, developed, and applied a convolutional neural network (CNN) architecture using multi-task learning to search for and characterize strong HI Lya absorption in quasar spectra. Without any explicit modeling of the quasar continuum nor application of the predicted line-profile for Lya from quantum mechanics, our algorithm predicts the presence of strong HI absorption and estimates the corresponding redshift zabs and HI column density NHI, with emphasis on damped Lya systems (DLAs, absorbers with log NHI > 20.3). We tuned the CNN model using a custom training set of DLAs injected into DLA-free quasar spectra from the Sloan Digital Sky Survey (SDSS), data release 5 (DR5). Testing on a held-back validation set demonstrates a high incidence of DLAs recovered by the algorithm (97.4% as DLAs and 99% as an HI absorber with log NHI > 19.5) and excellent estimates for zabs and NHI. Similar results are obtained against a human-generated survey of the SDSS DR5 dataset. The algorithm yields a low incidence of false positives and negatives but is challenged by overlapping DLAs and/or very high NHI systems. We have applied this CNN model to the quasar spectra of SDSS-DR7 and the Baryonic Oscillation Spectroscopic Survey (BOSS, data release 12) and provide catalogs of 4,913 and 50,969 DLAs respectively (including 1,659 and 9,230 high-confidence DLAs that were previously unpublished). This work validates the application of deep learning techniques to astronomical spectra for both classification and quantitative measurements.

20 pages, 19 figures; submitted to MNRAS; comments welcome; code available at https://github.com/davidparks21/qso_lya_detection_pipeline

References in corpus (29)

• Adam: A Method for Stochastic Optimization
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• A Catalog of Quasar Properties from SDSS DR7
• The Sloan Digital Sky Survey Quasar Catalog V. Seventh Data Release
• Damped Lyman alpha Systems
• The SDSS Damped Lya Survey: Data Release 3
• The Evolution of the Intergalactic Medium
• Column density distribution and cosmological mass density of neutral gas: Sloan Digital Sky Survey-III Data Release 9
• On the (Non)Evolution of HI Disks over Cosmic Time
• Evolution of the cosmological mass density of neutral gas from Sloan Digital Sky Survey II - Data Release 7
• The UCSD HIRES/KeckI Damped Lya Abundance Database: II. The Implications
• Dust in High Redshift Galaxies
• The Sloan Digital Sky Survey Quasar Catalog I. Early Data Release
• The Fundamental Plane of Damped Lyman Alpha Systems
• Metallicity Evolution in the Early Universe
• ALMA Observation of 158 micron [CII] Line and Dust Continuum of a z=7 Normally Star-forming Galaxy in the Epoch of Reionization
• A Keck HIRES Investigation of the Metal Abundances and Kinematics of Three Damped Lya Systems Toward Q2206-199
• IGM Constraints from the SDSS-III/BOSS DR9 Ly-alpha Forest Flux Probability Distribution Function
• MApping the Most Massive Overdensities Through Hydrogen (MAMMOTH) I: Methodology
• The SDSS Damped Lya Survey: Data Release 1
• [CII] 158 micron Emission from the Host Galaxies of Damped Lyman Alpha Systems
• A z = 3 Lyman Alpha Blob Associated with a Damped Lyman Alpha System Proximate to its Background Quasar
• Statistical properties of damped Lyman-alpha systems from Sloan Digital Sky Survey DR12
• The Keck+Magellan Survey for Lyman Limit Absorption III: Sample Definition and Column Density Measurements
• A glance at the host galaxy of high-redshift quasars using strong damped Lyman-alpha systems as coronagraphs
• Detecting Damped Lyman-$\alpha$ Absorbers with Gaussian Processes
• A Unique View of AGN-Driven Molecular Outflows: The Discovery of a Massive Galaxy Counterpart to a $z=2.4$ High-Metallicity Damped Lyman-$\alpha$ Absorber
• A Glimpse at Quasar Host Galaxy Far-UV Emission, Using DLAs as Natural Coronagraphs
• Asymmetric Deviation of the Scattering Cross Section around Ly$\alpha$ by Atomic Hydrogen

Cited by in corpus (53)

• The Sloan Digital Sky Survey Quasar Catalog: Sixteenth Data Release
• The Completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: Baryon acoustic oscillations with Lyman-$\alpha$ forests
• The Cosmic Baryon and Metal Cycles
• DESI 2024 IV: Baryon Acoustic Oscillations from the Lyman Alpha Forest
• Probabilistic Random Forest: A machine learning algorithm for noisy datasets
• The one-dimensional power spectrum from the SDSS DR14 Ly$\alpha$ forests
• Deep learning from 21-cm tomography of the Cosmic Dawn and Reionization
• The UVES Spectral Quasar Absorption Database (SQUAD) Data Release 1: The first 10 million seconds
• The Dark Energy Spectroscopic Instrument: One-dimensional power spectrum from first Lyman-$\alpha$ forest samples with Fast Fourier Transform
• Machine learning Applied to Star-Galaxy-QSO Classification and Stellar Effective Temperature Regression
• On Neural Architectures for Astronomical Time-series Classification with Application to Variable Stars
• Constraining the H${}_2$ column density distribution at z$\sim$3 from composite DLA spectra
• Classification of Broad Absorption Line Quasars with a Convolutional Neural Network
• A deep learning model to emulate simulations of cosmic reionization
• Deep Learning Based Detection of Cosmological Diffuse Radio Sources
• A tomographic map of the large-scale matter distribution using the eBOSS Stripe 82 Ly-$\alpha$ forest
• A Survey on Neural Network Interpretability
• Damped Lyman-alpha Absorbers from Sloan Digital Sky Survey DR16Q with Gaussian processes
• Machine learning initialization to accelerate Stokes profile inversions
• Cosmological Constraints from the eBOSS Lyman-$\alpha$ Forest using the PRIYA Simulations
• Mimicking spectropolarimetric inversions using convolutional neural networks
• Detecting Multiple DLAs per Spectrum in SDSS DR12 with Gaussian Processes
• Searching for AGN and Pulsar Candidates in 4FGL Unassociated Sources Using Machine Learning
• Quasar Factor Analysis -- An Unsupervised and Probabilistic Quasar Continuum Prediction Algorithm with Latent Factor Analysis
• Searching for hot subdwarf stars from the LAMOST spectra. III. classifying the hot subdwarf stars from LAMOST DR4 using deep learning method
• The Completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: The Damped Lyman-$\alpha$ systems Catalog
• Imprints of the first billion years: Lyman limit systems at $z\sim 5$
• deepSIP: Linking Type Ia Supernova Spectra to Photometric Quantities with Deep Learning
• Synthetic spectra for Lyman-$\alpha$ forest analysis in the Dark Energy Spectroscopic Instrument
• Identifying MgII Narrow Absorption Lines with Deep Learning
• The cross-correlation of galaxies in absorption with the Lyman $\alpha$ forest
• Deep Forest: Neural Network reconstruction of the Lyman-alpha forest
• MALS SALT-NOT survey of MIR-selected powerful radio-bright AGN at 0<z<3.5
• Constraining the Halo Mass of Damped Ly$\alpha$ Absorption Systems (DLAs) at $z=2-3.5$ using the Quasar-CMB Lensing Cross-correlation
• Fraction of broad absorption line quasars in different radio morphologies
• Metal Lines Associated with the Lyman-alpha Forest from eBOSS Data
• Galaxy Spectra neural Network (GaSNet). II. Using Deep Learning for Spectral Classification and Redshift Predictions
• Detecting neutral hydrogen at z > 3 in large spectroscopic surveys of quasars
• Redshifted broad absorption line quasars found via machine-learned spectral similarity
• Probing Large-scale UV Background Inhomogeneity Associated with Quasars using Metal Absorption
• Ly$\alpha$NNA: A Deep Learning Field-level Inference Machine for the Lyman-$\alpha$ Forest
• Searching for cold gas traced by MgII quasar absorbers in massive X-ray-selected galaxy clusters
• Using Machine Learning to Find Ghostly Damped Ly$\alpha$ Systems in SDSS DR14
• A survey of extremely metal-poor gas at cosmic noon: evidence of elevated [O/Fe]
• Discovering Ca II Absorption Lines With a Neural Network
• Automated Measurement of Quasar Redshift with a Gaussian Process
• Harvesting the Lyα forest with convolutional neural networks
• Molecular hydrogen in absorption at high redshifts. Science cases for CUBES
• 3D Correlations in the Lyman-$\alpha$ Forest from Early DESI Data
• PSF-based Analysis for Detecting Unresolved Wide Binaries
• CHILES X: Molecular and atomic gas at intermediate redshift
• DESI DR1 Lyα 1D power spectrum: The Fast Fourier Transform estimator measurement
• Efficient neutral-IGM inference from noisy 21-cm forest spectra with latent-space U-Net encoding and XGBoost