Quasar Factor Analysis -- An Unsupervised and Probabilistic Quasar Continuum Prediction Algorithm with Latent Factor Analysis

Abstract

Since their first discovery, quasars have been essential probes of the distant Universe. However, due to our limited knowledge of its nature, predicting the intrinsic quasar continua has bottlenecked their usage. Existing methods of quasar continuum recovery often rely on a limited number of high-quality quasar spectra, which might not capture the full diversity of the quasar population. In this study, we propose an unsupervised probabilistic model, Quasar Factor Analysis (QFA), which combines factor analysis (FA) with physical priors of the intergalactic medium (IGM) to overcome these limitations. QFA captures the posterior distribution of quasar continua through generatively modeling quasar spectra. We demonstrate that QFA can achieve the state-of-the-art performance, $\sim 2\%$ relative error, for continuum prediction in the Ly$\alpha$ forest region compared to previous methods. We further fit 90,678 $2<\mathrm{z}<3.5$, SNR$>2$ quasar spectra from Sloan Digital Sky Survey Data Release 16 and found that for $\sim 30\%$ quasar spectra where the continua were ill-determined with previous methods, QFA yields visually more plausible continua. QFA also attains $\lesssim 1\%$ error in the 1D Ly$\alpha$ power spectrum measurements at $\mathrm{z}\sim 3$ and $\sim 4\%$ in $\mathrm{z}\sim 2.4$. In addition, QFA determines latent factors representing more physically motivated than PCA. We investigate the evolution of the latent factors and report no significant redshift or luminosity dependency except for the Baldwin effect. The generative nature of QFA also enables outlier detection robustly; we showed that QFA is effective in selecting outlying quasar spectra, including damped Ly$\alpha$ systems and potential Type II quasar spectra.

Main body is 25 pages with 14 figures. Much more detailed exposition of the method originally presented in the short conference workshop paper arXiv:2207.02788 . All source codes are made publicly available at https://doi.org/10.5281/zenodo.8025967. Datasets in this work are publicly available at https://doi.org/10.5281/zenodo.8050660. Accepted by ApJS. Comments are welcome!

References in corpus (19)

• Array Programming with NumPy
• The Astropy Project: Sustaining and Growing a Community-oriented Open-source Project and the Latest Major Release (v5.0) of the Core Package
• A Catalog of Quasar Properties from SDSS DR7
• The Sloan Digital Sky Survey Quasar Catalog: Sixteenth Data Release
• The diversity of quasars unified by accretion and orientation
• The Completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: Baryon acoustic oscillations with Lyman-$\alpha$ forests
• Metallicity Evolution of Damped Lyman-alpha Systems out to z~5
• 4MOST: Project overview and information for the First Call for Proposals
• A Direct Precision Measurement of the Intergalactic Lyman-alpha Opacity at 2<z<4.2
• Target Selection and Validation of DESI Quasars
• New Constraints on Quasar Evolution: Broad Line Velocity Shifts over $1.5\lesssim z\lesssim 7.5$
• Cosmological inference from the EFTofLSS: the eBOSS QSO full-shape analysis
• Classification of Broad Absorption Line Quasars with a Convolutional Neural Network
• Optimal 1D Ly-$\alpha$ Forest Power Spectrum Estimation I: DESI-Lite Spectra
• Direct cosmological inference from three-dimensional correlations of the Lyman-$\alpha$ forest
• Joint Bayesian Estimation of Quasar Continua and the Lyman-Alpha Forest Flux Probability Distribution Function
• Impact of Massive Neutrinos and Dark Radiation on the High-redshift Cosmic Web. I. Lyman-Alpha Forest Observables
• Probing the Diffuse Lyman-alpha Emission on Cosmological Scales: Lyα Emission Intensity Mapping Using the Complete SDSS-IV eBOSS Survey
• A Generative Model for Quasar Spectra

Cited by in corpus (5)

• PopSED: Population-Level Inference for Galaxy Properties from Broadband Photometry with Neural Density Estimation
• The Qz5 Survey (I): How the HI Mass Density of the Universe Evolves With Cosmic Time
• Automated quasar continuum estimation using neural networks: a comparative study of deep-learning architectures
• Reionization relics in the cross-correlation between the Ly$\alpha$ forest and 21 cm intensity mapping in the post-reionization era
• Can AI Dream of Unseen Galaxies? Conditional Diffusion Model for Galaxy Morphology Augmentation