Automated data processing architecture for the Gemini Planet Imager Exoplanet Survey

Abstract

The Gemini Planet Imager Exoplanet Survey (GPIES) is a multi-year direct imaging survey of 600 stars to discover and characterize young Jovian exoplanets and their environments. We have developed an automated data architecture to process and index all data related to the survey uniformly. An automated and flexible data processing framework, which we term the Data Cruncher, combines multiple data reduction pipelines together to process all spectroscopic, polarimetric, and calibration data taken with GPIES. With no human intervention, fully reduced and calibrated data products are available less than an hour after the data are taken to expedite follow-up on potential objects of interest. The Data Cruncher can run on a supercomputer to reprocess all GPIES data in a single day as improvements are made to our data reduction pipelines. A backend MySQL database indexes all files, which are synced to the cloud, and a front-end web server allows for easy browsing of all files associated with GPIES. To help observers, quicklook displays show reduced data as they are processed in real-time, and chatbots on Slack post observing information as well as reduced data products. Together, the GPIES automated data processing architecture reduces our workload, provides real-time data reduction, optimizes our observing strategy, and maintains a homogeneously reduced dataset to study planet occurrence and instrument performance.

21 pages, 3 figures, accepted in JATIS

References in corpus (21)

• Astropy: A Community Python Package for Astronomy
• Detection and Characterization of Exoplanets and Disks using Projections on Karhunen-Loeve Eigenimages
• The Gemini Planet Imager: First Light
• Fundamental limitations of high contrast imaging set by small sample statistics
• Polarimetry with the Gemini Planet Imager: Methods, Performance at First Light, and the Circumstellar Ring around HR 4796A
• Improving and Assessing Planet Sensitivity of the GPI Exoplanet Survey with a Forward Model Matched Filter
• Reconnaissance of the HR 8799 Exosolar System II: Astrometry and Orbital Motion
• Astrometry and Photometry with Coronagraphs
• Experimental Design for the Gemini Planet Imager
• Discovery of a Substellar Companion to the Nearby Debris Disk Host HR 2562
• Gemini Planet Imager Observational Calibrations I: Overview of the GPI Data Reduction Pipeline
• Gemini Planet Imager Observational Calibrations VIII: Characterization and Role of Satellite Spots
• Gemini Planet Imager Observational Calibrations V: Astrometry and Distortion
• GPI PSF subtraction with TLOCI: the next evolution in exoplanet/disk high-contrast imaging
• Gemini Planet Imager Observational Calibrations IV: Wavelength Calibration and Flexure Correction for the Integral Field Spectrograph
• Computer vision applications for coronagraphic optical alignment and image processing
• Status and performance of the Gemini Planet Imager adaptive optics system
• The Integral Field Spectrograph for the Gemini Planet Imager
• Gemini Planet Imager Observational Calibrations II: Detector Performance and Calibration
• Gemini Planet Imager Observational Calibrations XIV: Polarimetric Contrasts and New Data Reduction Techniques
• Ground-Based Direct Detection of Exoplanets with the Gemini Planet Imager (GPI)

Cited by in corpus (15)

• The Gemini Planet Imager Exoplanet Survey: Giant Planet and Brown Dwarf Demographics From 10-100 AU
• TESS Hunt for Young and Maturing Exoplanets (THYME): A planet in the 45 Myr Tucana-Horologium association
• Dynamical Constraints on the HR 8799 Planets with GPI
• GPI spectra of HR 8799 c, d, and e from 1.5 to 2.4$\mu$m with KLIP Forward Modeling
• The Gemini Planet Imager Exoplanet Survey: Dynamical Mass of the Exoplanet beta Pictoris b from Combined Direct Imaging and Astrometry
• An Exo-Kuiper Belt and An Extended Halo around HD 191089 in Scattered Light
• Gemini Planet Imager Spectroscopy of the Dusty Substellar Companion HD 206893 B
• An updated visual orbit of the directly-imaged exoplanet 51 Eridani b and prospects for a dynamical mass measurement with Gaia
• First Resolved Scattered-Light Images of Four Debris Disks in Scorpius-Centaurus with the Gemini Planet Imager
• Exoplanet Imaging Data Challenge: benchmarking the various image processing methods for exoplanet detection
• A low-mass companion desert among intermediate-mass visual binaries: The scaled-up counterpart to the brown dwarf desert
• A deep search for planets in the inner 15 au around Vega
• Mining the GPIES database
• HD 143811 AB b: A Directly Imaged Planet Orbiting a Spectroscopic Binary in Sco-Cen
• Analyzing the Atmospheric Dispersion Correction of the Gemini Planet Imager: residual dispersion above design requirements