Quasar Factor Analysis -- An Unsupervised and Probabilistic Quasar Continuum Prediction Algorithm with Latent Factor Analysis

Abstract

Since their first discovery, quasars have been essential probes of the distant Universe. However, due to our limited knowledge of its nature, predicting the intrinsic quasar continua has bottlenecked their usage. Existing methods of quasar continuum recovery often rely on a limited number of high-quality quasar spectra, which might not capture the full diversity of the quasar population. In this study, we propose an unsupervised probabilistic model, Quasar Factor Analysis (QFA), which combines factor analysis (FA) with physical priors of the intergalactic medium (IGM) to overcome these limitations. QFA captures the posterior distribution of quasar continua through generatively modeling quasar spectra. We demonstrate that QFA can achieve the state-of-the-art performance, $\sim 2\%$ relative error, for continuum prediction in the Ly$\alpha$ forest region compared to previous methods. We further fit 90,678 $2<\mathrm{z}<3.5$, SNR$>2$ quasar spectra from Sloan Digital Sky Survey Data Release 16 and found that for $\sim 30\%$ quasar spectra where the continua were ill-determined with previous methods, QFA yields visually more plausible continua. QFA also attains $\lesssim 1\%$ error in the 1D Ly$\alpha$ power spectrum measurements at $\mathrm{z}\sim 3$ and $\sim 4\%$ in $\mathrm{z}\sim 2.4$. In addition, QFA determines latent factors representing more physically motivated than PCA. We investigate the evolution of the latent factors and report no significant redshift or luminosity dependency except for the Baldwin effect. The generative nature of QFA also enables outlier detection robustly; we showed that QFA is effective in selecting outlying quasar spectra, including damped Ly$\alpha$ systems and potential Type II quasar spectra.

arXiv commentMain body is 25 pages with 14 figures. Much more detailed exposition of the method originally presented in the short conference workshop paper arXiv:2207.02788 . All source codes are made publicly available at https://doi.org/10.5281/zenodo.8025967. Datasets in this work are publicly available at https://doi.org/10.5281/zenodo.8050660. Accepted by ApJS. Comments are welcome!

References in corpus (64)

• Array Programming with NumPy
• The Astropy Project: Sustaining and Growing a Community-oriented Open-source Project and the Latest Major Release (v5.0) of the Core Package
• A Catalog of Quasar Properties from SDSS DR7
• Constraining the Evolution of the Ionizing Background and the Epoch of Reionization with z ~ 6 Quasars II: A Sample of 19 Quasars
• Determining Central Black Hole Masses in Distant Active Galaxies and Quasars. II. Improved Optical and UV Scaling Relationships
• Evidence for Reionization at z ~ 6: Detection of a Gunn-Peterson Trough in a z=6.28 Quasar
• Equation of State of the Photoionized Intergalactic Medium
• Damped Lyman alpha Systems
• Quasar-Lyman $\alpha$ Forest Cross-Correlation from BOSS DR11 : Baryon Acoustic Oscillations
• The Sloan Digital Sky Survey Quasar Catalog: Sixteenth Data Release
• First constraints on fuzzy dark matter from Lyman-$\alpha$ forest data and hydrodynamical simulations
• The diversity of quasars unified by accretion and orientation
• The Completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: Baryon acoustic oscillations with Lyman-$\alpha$ forests
• Metallicity Evolution of Damped Lyman-alpha Systems out to z~5
• Quantitative Constraints on the Reionization History from the IGM Damping Wing Signature in Two Quasars at z > 7
• 4MOST: Project overview and information for the First Call for Proposals
• The origin of the Broad Line Region in Active Galactic Nuclei
• Measurement of Baryon Acoustic Oscillations in the Lyman-alpha Forest Fluctuations in BOSS Data Release 9
• A Direct Precision Measurement of the Intergalactic Lyman-alpha Opacity at 2<z<4.2
• The one-dimensional Ly-alpha forest power spectrum from BOSS
• Target Selection and Validation of DESI Quasars
• A Refined Measurement of the Mean Transmitted Flux in the Ly-alpha Forest over 2 < z < 5 Using Composite Quasar Spectra
• Constraints on neutrino masses from Lyman-alpha forest power spectrum with BOSS and XQ-100
• Constraints on reionisation from the z=7.5 QSO ULASJ1342+0928
• The one-dimensional power spectrum from the SDSS DR14 Ly$\alpha$ forests
• The Rapid Decline in Metallicity of Damped Ly-alpha Systems at z~5
• Measurements of the z ~ 6 Intergalactic Medium Optical Depth and Transmission Spikes Using a New z > 6.3 Quasar Sample
• A Principal Component Analysis of quasar UV spectra at z~3
• Ionization near-zones associated with quasars at z ~ 6
• A consistent and robust measurement of the thermal state of the IGM at $2\le z\le 4$ from a large sample of Ly$\alpha$ forest spectra: Evidence for late and rapid HeII reionization
• The H I opacity of the intergalactic medium at redshifts 1.6 < z < 3.2
• SuperRAENN: A Semi-supervised Supernova Photometric Classification Pipeline Trained on Pan-STARRS1 Medium Deep Survey Supernovae
• Predicting QSO Continua in the Ly Alpha Forest
• Deep Learning of Quasar Spectra to Discover and Characterize Damped Lya Systems
• Predicting Quasar Continua Near Lyman-$\alpha$ with Principal Component Analysis
• Mean-flux Regulated PCA Continuum Fitting of SDSS Lyman-alpha Forest Spectra
• The Lyman-$\alpha$ forest as a diagnostic of the nature of the dark matter
• New Constraints on Quasar Evolution: Broad Line Velocity Shifts over $1.5\lesssim z\lesssim 7.5$
• Reionization history constraints from neural network based predictions of high-redshift quasar continua
• Intensity mapping with SDSS/BOSS Lyman-alpha emission, quasars and their Lyman-alpha forest
• A comparison of quasar emission reconstruction techniques for $z\ge 5.0$ Lyman-$\alpha$ and Lyman-$\beta$ transmission
• LyaCoLoRe: Synthetic Datasets for Current and Future Lyman-$\alpha$ Forest BAO Surveys
• Classification of Broad Absorption Line Quasars with a Convolutional Neural Network
• Cosmological inference from the EFTofLSS: the eBOSS QSO full-shape analysis
• A glance at the host galaxy of high-redshift quasars using strong damped Lyman-alpha systems as coronagraphs
• Constraints on dark radiation from cosmological probes
• Angular clustering properties of the DESI QSO target selection using DR9 Legacy Imaging Surveys
• Optimal 1D Ly-$\alpha$ Forest Power Spectrum Estimation I: DESI-Lite Spectra
• Mock Quasar-Lyman-α Forest Data-sets for the SDSS-III Baryon Oscillation Spectroscopic Survey
• Cycle-StarNet: Bridging the gap between theory and data by leveraging large datasets
• Spectral Evolution in High Redshift Quasars from the Final BOSS Sample
• Detecting Damped Lyman-$\alpha$ Absorbers with Gaussian Processes
• Systematic Continuum Errors in the Lyman-Alpha Forest and The Measured Temperature-Density Relation
• A semi-supervised Machine Learning search for never-seen Gravitational-Wave sources
• A Deep Learning Approach to Quasar Continuum Prediction
• Detecting Multiple DLAs per Spectrum in SDSS DR12 with Gaussian Processes
• The PAU survey: Ly$\alpha$ intensity mapping forecast
• Direct cosmological inference from three-dimensional correlations of the Lyman-$\alpha$ forest
• Measurements of effective optical depth in the Ly$\alpha$ forest from the BOSS DR12 Quasar sample
• Ly$\alpha$ forest power spectrum as an emerging window into the epoch of reionization and cosmic dawn
• Impact of Massive Neutrinos and Dark Radiation on the High-redshift Cosmic Web. I. Lyman-Alpha Forest Observables
• Joint Bayesian Estimation of Quasar Continua and the Lyman-Alpha Forest Flux Probability Distribution Function
• Probing the Diffuse Lyman-alpha Emission on Cosmological Scales: Lyα Emission Intensity Mapping Using the Complete SDSS-IV eBOSS Survey
• A Generative Model for Quasar Spectra

Cited by in corpus (5)

• PopSED: Population-Level Inference for Galaxy Properties from Broadband Photometry with Neural Density Estimation
• The Qz5 Survey (I): How the HI Mass Density of the Universe Evolves With Cosmic Time
• Automated quasar continuum estimation using neural networks: a comparative study of deep-learning architectures
• Reionization relics in the cross-correlation between the Ly$\alpha$ forest and 21 cm intensity mapping in the post-reionization era
• Can AI Dream of Unseen Galaxies? Conditional Diffusion Model for Galaxy Morphology Augmentation