The ALMA Interferometric Pipeline Heuristics

Abstract

We describe the calibration and imaging heuristics developed and deployed in the ALMA interferometric data processing pipeline, as of ALMA Cycle 9. The pipeline software framework is written in Python, with each data reduction stage layered on top of tasks and toolkit functions provided by the Common Astronomy Software Applications package. This framework supports a variety of tasks for observatory operations, including science data quality assurance, observing mode commissioning, and user reprocessing. It supports ALMA and VLA interferometric data along with ALMA and NRO45m single dish data, via different stages and heuristics. In addition to producing calibration tables, calibrated measurement sets, and cleaned images, the pipeline creates a WebLog which serves as the primary interface for verifying the data quality assurance by the observatory and for examining the contents of the data by the user. Following the adoption of the pipeline by ALMA Operations in 2014, the heuristics have been refined through annual development cycles, culminating in a new pipeline release aligned with the start of each ALMA Cycle of observations. Initial development focused on basic calibration and flagging heuristics (Cycles 2-3), followed by imaging heuristics (Cycles 4-5), refinement of the flagging and imaging heuristics with parallel processing (Cycles 6-7), addition of the moment difference analysis to improve continuum channel identification (2020 release), addition of a spectral renormalization stage (Cycle 8), and improvement in low SNR calibration heuristics (Cycle 9). In the two most recent Cycles, 97% of ALMA datasets were calibrated and imaged with the pipeline, ensuring long-term automated reproducibility. We conclude with a brief description of plans for future additions, including self-calibration, multi-configuration imaging, and calibration and imaging of full polarization data.

arXiv commentAccepted for publication by Publications of the Astronomical Society of the Pacific, 65 pages, 20 figures, 10 tables, 2 appendices. Small corrections and additions from the proof stage have been applied

References in corpus (21)

• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• Array Programming with NumPy
• The Astropy Project: Sustaining and Growing a Community-oriented Open-source Project and the Latest Major Release (v5.0) of the Core Package
• CASA, the Common Astronomy Software Applications for Radio Astronomy
• A multi-scale multi-frequency deconvolution algorithm for synthesis imaging in radio interferometry
• The Atacama Large Millimeter/submillimeter Array
• PHANGS-ALMA Data Processing and Pipeline
• AUTO-MULTITHRESH: A General Purpose Automasking Algorithm
• Digital Spectro-Correlator System for the Atacama Compact Array of the Atacama Large Millimeter/submillimeter Array
• ALMACAL IV: A catalogue of ALMA calibrator continuum observations
• Calibration of ALMA as a phased array: ALMA observations during the 2017 VLBI campaign
• Phase Correction for ALMA with 183 GHz Water Vapour Radiometers
• EHT-HOPS pipeline for millimeter VLBI data reduction
• ALMA Long Baseline Campaigns: Phase Characteristics of Atmosphere at Long Baselines in the Millimeter and Submillimeter Wavelengths
• The Atacama Large Millimeter/submillimeter Array: Overview & status
• Data Combination: Interferometry and Single-dish Imaging in Radio Astronomy
• ALMA observations of RCW 120 Fragmentation at 0.01pc scale
• The JCMT Gould Belt Survey: the effect of molecular contamination in SCUBA-2 observations of Orion A
• Analysis of antenna position measurements and weather station network data during the ALMA Long Baseline Campaign of 2015
• ALMA High-frequency Long-baseline Campaign in 2017: A Comparison of the Band-to-band and In-band Phase Calibration Techniques and Phase-calibrator Separation Angles
• The ALMA archive and its place in the astronomy of the future

Cited by in corpus (29)

• Isotopic evidence of long-lived volcanism on Io
• Joint ALMA/X-ray monitoring of the radio-quiet type 1 AGN IC 4329A
• ASW${}^2$DF: Census of the obscured star formation in a galaxy cluster in formation at $z=2.2$
• One month convection timescale on the surface of a giant evolved star
• Dual-band Unified Exploration of Three CMZ Clouds (DUET). Cloud-wide census of continuum sources showing low spectral indices
• ALMA polarimetry of radio-quiet AGNs
• Measuring 60-pc-scale Star Formation Rate of the Nearby Seyfert Galaxy NGC 1068 with ALMA, HST, VLT/MUSE, and VLA
• ALMA observation of evolving magnetized corona in the radio-quiet changing-state AGN NGC 1566
• A [CII]-158-micron Survey of DLA galaxies at z~4: Observations of Dense and Metal-Enriched Neutral Gas within the Circumgalactic Medium of Star-Forming Galaxies
• Quasar radiation transforms the gas in a merging companion galaxy
• Formation of Substructure in Luminous Submillimeter Galaxies (FOSSILS): Evidence of Multiple Pathways to Trigger Starbursts in Luminous Submillimeter Galaxies
• PASSAGES: The Discovery of a Strongly Lensed Protocluster Core Candidate at Cosmic Noon
• ALMA survey of a massive node of the Cosmic Web at $z\sim 3$. II. A dynamically cold and massive disk galaxy in the proximity of a hyperluminous quasar
• ALMA discovery of Punctum -- a highly polarized mm source in nuclear starburst galaxy NGC 4945
• Reinforcement Learning for Data-Driven Workflows in Radio Interferometry. I. Principal Demonstration in Calibration
• The pre-eruption state of T CrB as observed with ALMA in 2024
• Accelerating the CLEAN algorithm of radio interferometry with convex optimization
• CHILES X: Molecular and atomic gas at intermediate redshift
• Investigating the role of turbulence in the interstellar medium in $z\sim 3$ dusty star-forming galaxies using kpc-resolution ALMA dust and gas maps
• The CHIMERA Survey: The first CO detection in Leo T, the lowest mass known galaxy still hosting cold molecular gas
• ALMAGAL VIII. Early phases of triggered star formation in source AG286.0716$-$1.8229
• Revisiting the Gas Dynamics of Henize 2-10: Possible Drivers of the Starburst
• The fragmentation properties of massive star-forming regions in 30Dor-10 at 2000 au resolution
• First Detection of HC5N in a Class II Disk around TW Hya
• The ${}^{13}\mathrm{CO}(2-1){/}^{12}\mathrm{CO}(2-1)$ Line Ratio from 100 Molecular Clouds in the Large Magellanic Cloud
• Resolving circumgalactic gas flows around a z$\approx$3.6 quasar using MUSE and ALMA
• The 107 GHz methanol transition is a dasar in G0.253+0.016
• Uncovering the absorbed atomic Universe with the [OI]63um line
• A deep ALMA Band 3 survey of HDFS/MUSE3D: Survey description and initial results