The Spectroscopic Data Processing Pipeline for the Dark Energy Spectroscopic Instrument

Abstract

We describe the spectroscopic data processing pipeline of the Dark Energy Spectroscopic Instrument (DESI), which is conducting a redshift survey of about 40 million galaxies and quasars using a purpose-built instrument on the 4-m Mayall Telescope at Kitt Peak National Observatory. The main goal of DESI is to measure with unprecedented precision the expansion history of the Universe with the Baryon Acoustic Oscillation technique and the growth rate of structure with Redshift Space Distortions. Ten spectrographs with three cameras each disperse the light from 5000 fibers onto 30 CCDs, covering the near UV to near infrared (3600 to 9800 Angstrom) with a spectral resolution ranging from 2000 to 5000. The DESI data pipeline generates wavelength- and flux-calibrated spectra of all the targets, along with spectroscopic classifications and redshift measurements. Fully processed data from each night are typically available to the DESI collaboration the following morning. We give details about the pipeline's algorithms, and provide performance results on the stability of the optics, the quality of the sky background subtraction, and the precision and accuracy of the instrumental calibration. This pipeline has been used to process the DESI Survey Validation data set, and has exceeded the project's requirements for redshift performance, with high efficiency and a purity greater than 99 percent for all target classes.

AJ, revised version, 55 pages, 55 figures, 4 tables

References in corpus (12)

• K-corrections and filter transformations in the ultraviolet, optical, and near infrared
• The DESI Experiment Part I: Science,Targeting, and Survey Design
• Overview of the Instrumentation for the Dark Energy Spectroscopic Instrument
• Galaxy And Mass Assembly (GAMA): Data Release 4 and the z < 0.1 total and z < 0.08 morphological galaxy stellar mass functions
• Target Selection and Validation of DESI Luminous Red Galaxies
• The Robotic Multi-Object Focal Plane System of the Dark Energy Spectroscopic Instrument (DESI)
• The Completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: Large-scale Structure Catalogs for Cosmological Analysis
• The Target-selection Pipeline for the Dark Energy Spectroscopic Instrument
• Survey of Surveys I: The largest catalogue of radial velocities for the Galaxy
• Redshift Measurement and Spectral Classification for eBOSS Galaxies with the Redmonster Software
• Nonnegative Matrix Factorization (NMF) with Heteroscedastic Uncertainties and Missing data
• Performance of Kitt Peak's Mayall 4-meter Telescope During DESI Commissioning

Cited by in corpus (92)

• DESI 2024 VI: Cosmological Constraints from the Measurements of Baryon Acoustic Oscillations
• DESI DR2 Results II: Measurements of Baryon Acoustic Oscillations and Cosmological Constraints
• The Early Data Release of the Dark Energy Spectroscopic Instrument
• DESI 2024 III: Baryon Acoustic Oscillations from Galaxies and Quasars
• DESI Bright Galaxy Survey: Final Target Selection, Design, and Validation
• DESI 2024 IV: Baryon Acoustic Oscillations from the Lyman Alpha Forest
• DESI 2024 VII: Cosmological Constraints from the Full-Shape Modeling of Clustering Measurements
• DESI 2024 II: Sample Definitions, Characteristics, and Two-point Clustering Statistics
• DESI 2024 V: Full-Shape Galaxy Clustering from Galaxies and Quasars
• Dynamical Dark Energy in light of the DESI DR2 Baryonic Acoustic Oscillations Measurements
• Performance of the Quasar Spectral Templates for the Dark Energy Spectroscopic Instrument
• The Dark Energy Spectroscopic Instrument: One-dimensional power spectrum from first Lyman-$\alpha$ forest samples with Fast Fourier Transform
• Modified Gravity Constraints from the Full Shape Modeling of Clustering Measurements from DESI 2024
• First Detection of the BAO Signal from Early DESI Data
• Target Selection for the DESI Peculiar Velocity Survey
• DESI z >~ 5 Quasar Survey. I. A First Sample of 400 New Quasars at z ~ 4.7-6.6
• The Atacama Cosmology Telescope DR6 and DESI: Structure formation over cosmic time with a measurement of the cross-correlation of CMB Lensing and Luminous Red Galaxies
• Value Added Catalog of physical properties of more than 1.3 million galaxies from the DESI Survey
• DESI Early Data Release Milky Way Survey Value-Added Catalogue
• Mitigation of DESI fiber assignment incompleteness effect on two-point clustering with small angular scale truncated estimators
• Archetype-Based Redshift Estimation for the Dark Energy Spectroscopic Instrument Survey
• DESI Peculiar Velocity Survey -- Fundamental Plane
• Detecting and Characterizing Mg II absorption in DESI Survey Validation Quasar Spectra
• Analytical and EZmock covariance validation for the DESI 2024 results
• Cosmological constraints from the cross-correlation of DESI Luminous Red Galaxies with CMB lensing from Planck PR4 and ACT DR6
• Long-term follow-up observations of extreme coronal line emitting galaxies
• Fiducial-Cosmology-dependent systematics for the DESI 2024 BAO Analysis
• Measuring ${\sigma }_8$ using DESI Legacy Imaging Surveys Emission-Line Galaxies and Planck CMB Lensing and the Impact of Dust on Parameter Inference
• Exploring the interaction between the MW and LMC with a large sample of blue horizontal branch stars from the DESI survey
• Magellan/M2FS and MMT/Hectochelle Spectroscopy of Dwarf Galaxies and Faint Star Clusters within the Galactic Halo
• ELG Spectroscopic Systematics Analysis of the DESI Data Release 1
• Backlighting extended gas halos around luminous red galaxies: kinematic Sunyaev-Zel'dovich effect from DESI Y1 x ACT
• The most distant HI galaxies discovered by the 500 m dish FAST
• Positive neutrino masses with DESI DR2 via matter conversion to dark energy
• Synthetic spectra for Lyman-$\alpha$ forest analysis in the Dark Energy Spectroscopic Instrument
• Wide Area VISTA Extra-galactic Survey (WAVES): Unsupervised star-galaxy separation on the WAVES-Wide photometric input catalogue using UMAP and $\mathrm{HDBSCAN}$
• A framework to measure the properties of intergalactic metal systems with two-point flux statistics
• High redshift LBGs from deep broadband imaging for future spectroscopic surveys
• DESI Complete Calibration of the Color-Redshift Relation (DC3R2): Results from early DESI data
• The rate of extreme coronal line emitting galaxies in the Sloan Digital Sky Survey and their relation to tidal disruption events
• Constraints on primordial non-Gaussianity from the cross-correlation of DESI Luminous Red Galaxies and $Planck$ CMB lensing
• A Spectroscopic Search for Optical Emission Lines from Dark Matter Decay
• The DESI One-Percent Survey: Modelling the clustering and halo occupation of all four DESI tracers with Uchuu
• DESI Survey Validation Spectra Reveal an Increasing Fraction of Recently Quenched Galaxies at $z\sim 1$
• Cosmology from Planck CMB Lensing and DESI DR1 Quasar Tomography
• Model-Independent Measurement of the Matter-Radiation Equality Scale in DESI 2024
• DESI Massive Post-Starburst Galaxies at $\mathbf{z}\sim 1.2$ have compact structures and dense cores
• PAC in DESI. I. Galaxy Stellar Mass Function into the $10^6{\mathrm{M}}_{\odot }$ Frontier
• Galaxy Spectra Networks (GaSNet). III. Generative pre-trained network for spectrum reconstruction, redshift estimate and anomaly detection
• AT 2018dyk: tidal disruption event or active galactic nucleus? Follow-up observations of an extreme coronal line emitter with the Dark Energy Spectroscopic Instrument
• Emission Line Predictions for Mock Galaxy Catalogues: a New Differentiable and Empirical Mapping from DESI
• DESIVAST: A Catalog of Low-Redshift Voids using Data from the DESI DR1 Bright Galaxy Survey
• FNet II: Spectral Classification of Quasars, Galaxies, Stars, and broad absorption line (BAL) Quasars
• Beyond Traditional Diagnostics: Identifying Active Galactic Nuclei with Spectral Energy Distribution Fitting in DESI Data
• Nearby stellar substructures in the Galactic halo from DESI Milky Way Survey Year 1 Data Release
• Window convolution of the galaxy clustering bispectrum
• Sixteen Multiple-Amplifier Sensing Charge-Coupled Devices and Characterization Techniques Targeting the Next Generation of Astronomical Instruments
• The Compilation and Validation of the Spectroscopic Redshift Catalogs for the DESI-COSMOS and DESI-XMMLSS Fields
• DESI Mock Challenge: Constructing DESI galaxy catalogues based on FastPM simulations
• Exploring HOD-dependent systematics for the DESI 2024 Full-Shape galaxy clustering analysis
• $H_0$ Without the Sound Horizon (or Supernovae): A 2% Measurement in DESI DR1
• SpecPT (Spectroscopy Pre-trained Transformer) Model for Extragalactic Spectroscopy: I. Architecture and Automated Redshift Measurement
• Multi-Amplifier Sensing Charge-coupled Devices for Next Generation Spectroscopy
• Doppler bias: impact of peculiar velocities on color selection and the large scale structure of galaxy surveys
• Catalog-level blinding on the bispectrum for DESI-like galaxy surveys
• Reionization relics in the cross-correlation between the Ly$\alpha$ forest and 21 cm intensity mapping in the post-reionization era
• 3D Correlations in the Lyman-$\alpha$ Forest from Early DESI Data
• Interacting Dark Energy and Its Implications for Unified Dark Sector
• Studying baryon acoustic oscillations using photometric redshifts from the DESI Legacy Imaging survey DR9
• The Cosmic Evolution of CIV Absorbers at $1.4<z<4.5$: Insights from $100,000$ Systems in DESI Quasars
• DESI Emission-line Galaxies: Clustering Dependence on Stellar Mass and [OII] Luminosity
• Candidate strongly lensed type Ia supernovae in the Zwicky Transient Facility archive
• Constraining primordial non-Gaussianity from the large scale structure two-point and three-point correlation functions
• Inference of matter power spectrum at z=0 using DESI DR1 Full-Shape data
• New $H(z)$ measurement at Redshift = 0.12 with DESI Data Release 1
• Optimizing Charge-coupled Device Readout Enabled by the Floating-Gate Amplifier
• The Impact of Spectroscopic Redshift Errors on Cosmological Measurements
• Review of detector requirements: some challenges for the present
• The DESI DR1 Peculiar Velocity Survey: global zero-point and $H_0$ constraints
• DESI Mg II Absorbers: Extinction Characteristics & Quasar Redshift Accuracy
• The Lyman-$\alpha$ Forest from LBGs: First 3D Correlation Measurement with DESI and Prospects for Cosmology
• Galaxies at the edges: a complete census of MACS J0416.1-2403 cluster
• STag II: Classification of Serendipitous Supernovae Observed by Galaxy Redshift Surveys
• Searching for Type Ia Supernovae in the Dark Energy Spectroscopic Instrument
• Quiescent Host Galaxies of Extended Quasars Revealed by Spectrophotometric Decomposition
• Early results in the search for extreme coronal line emitters with the Dark Energy Spectroscopic Instrument
• Identifying Anomalous DESI Galaxy Spectra with a Variational Autoencoder
• Clustering analysis of medium-band selected high-redshift galaxies
• DESI DR1 Lyα 1D power spectrum: The Fast Fourier Transform estimator measurement
• Measurements of quasar proximity zones with the Ly$\alpha$ forest of DESI Y1 quasars
• Highly Efficient Selection of High-Redshift Emission-Line Galaxies for future DESI-like surveys with Deep Multi-band Imaging
• Parity-odd Four-Point Correlation Function from DESI Data Release 1 Luminous Red Galaxy Sample