Gaia Data Release 2: processing of the photometric data

Abstract

The second Gaia data release is based on 22 months of mission data with an average of 0.9 billion individual CCD observations per day. A data volume of this size and granularity requires a robust and reliable but still flexible system to achieve the demanding accuracy and precision constraints that Gaia is capable of delivering. The internal Gaia photometric system was initialised using an iterative process that is solely based on Gaia data. A set of calibrations was derived for the entire Gaia DR2 baseline and then used to produce the final mean source photometry. The photometric catalogue contains 2.5 billion sources comprised of three different grades depending on the availability of colour information and the procedure used to calibrate them: 1.5 billion gold, 144 million silver, and 0.9 billion bronze. These figures reflect the results of the photometric processing; the content of the data release will be different due to the validation and data quality filters applied during the catalogue preparation. The photometric processing pipeline, PhotPipe, implements all the processing and calibration workflows in terms of Map/Reduce jobs based on the Hadoop platform. This is the first example of a processing system for a large astrophysical survey project to make use of these technologies. The improvements in the generation of the integrated G-band fluxes, in the attitude modelling, in the cross-matching, and and in the identification of spurious detections led to a much cleaner input stream for the photometric processing. This, combined with the improvements in the definition of the internal photometric system and calibration flow, produced high-quality photometry. Hadoop proved to be an excellent platform choice for the implementation of PhotPipe in terms of overall performance, scalability, downtime, and manpower required for operations and maintenance.

References in corpus (14)

• The Gaia mission
• HEALPix -- a Framework for High Resolution Discretization, and Fast Analysis of Data Distributed on the Sphere
• Gaia Data Release 1. Summary of the astrometric, photometric, and survey properties
• Gaia Data Release 2: The astrometric solution
• Gaia Data Release 2: Photometric content and validation
• Gaia Data Release 2: Catalogue validation
• Gaia Data Release 1: Astrometry - one billion positions, two million proper motions and parallaxes
• The astrometric core solution for the Gaia mission. Overview of models, algorithms and software implementation
• Gaia Data Release 1, Pre-processing and source list creation
• Gaia data release 1: Principles of the photometric calibration of the G band
• Detecting stars, galaxies, and asteroids with Gaia
• The Gaia spectrophotometric standard stars survey. I. Preliminary results
• Gaia Data Release 2. Calibration and mitigation of electronic offset effects in the data
• Gaia Data Release 1: Validation of the photometry

Cited by in corpus (111)

• Gaia Data Release 2. Summary of the contents and survey properties
• Gaia Data Release 2: The astrometric solution
• Gaia Early Data Release 3: Photometric content and validation
• Gaia Data Release 2: Photometric content and validation
• Gaia Data Release 2: Catalogue validation
• Gaia Data Release 2: first stellar parameters from Apsis
• Gaia Data Release 3 Properties and validation of the radial velocities
• Gaia Early Data Release 3: The Gaia Catalogue of Nearby Stars
• The PHANGS-MUSE survey -- Probing the chemo-dynamical evolution of disc galaxies
• Gaia Data Release 3: Processing and validation of BP/RP low-resolution spectral data
• Gaia Data Release 2: Properties and validation of the radial velocities
• Gaia Data Release 2: The Celestial reference frame (Gaia-CRF2)
• Confirmation of the $Gaia$ DR2 parallax zero-point offset using asteroseismology and spectroscopy in the $Kepler$ field
• Distances and parallax bias in Gaia DR2
• Isochrone ages for ~3 million stars with the second Gaia data release
• A cooling anomaly of high-mass white dwarfs
• Gaia Photometric Science Alerts
• BANYAN. XIII. A First Look at Nearby Young Associations with Gaia Data Release 2
• The LOFAR Two Metre Sky Survey: Deep Fields Data Release 1 -- III. Host-galaxy identifications and value added catalogues
• I Spy Transits and Pulsations: Empirical Variability in White Dwarfs Using Gaia and the Zwicky Transient Facility
• Gaia Data Release 2: Observations of solar system objects
• A 5D view of the Alpha Per, Pleiades, and Praesepe clusters
• Gravitationally lensed quasars in Gaia: III. 22 new lensed quasars from Gaia Data Release 2
• Using Gaia DR2 to Constrain Local Dark Matter Density and Thin Dark Disk
• The GALAH survey: effective temperature calibration from the InfraRed Flux Method in the Gaia system
• Globular cluster number density profiles using Gaia DR2
• Homogeneous study of Herbig Ae/Be stars from spectral energy distributions and Gaia EDR3
• Catalogues of Active Galactic Nuclei From Gaia and unWISE Data
• New ultra-cool and brown dwarf candidates in Gaia DR2
• Double white dwarf merger products among high-mass white dwarfs
• Varstrometry for Off-nucleus and Dual sub-Kpc AGN (VODKA): Hubble Space Telescope Discovers Double Quasars
• Unveiling the Distinct Formation Pathways of the Inner and Outer Discs of the Milky Way with Bayesian Machine Learning
• WASP-186 and WASP-187: two hot Jupiters discovered by SuperWASP and SOPHIE with additional observations by TESS
• Nebular Spectra of 111 Type Ia Supernovae Disfavor Single Degenerate Progenitors
• Alone but not lonely: Observational evidence that binary interaction is always required to form hot subdwarf stars
• Stellar Rotation in the Gaia Era: Revised Open Clusters Sequences
• New light on the Gaia DR2 parallax zero-point: influence of the asteroseismic approach, in and beyond the Kepler field
• The Gaia DR2 parallax zero point: Hierarchical modeling of red clump stars
• Lessons from the curious case of the `fastest' star in Gaia DR2
• Microarcsecond Astrometry: Science Highlights from Gaia
• The EREBOS project -- Investigating the effect of substellar and low-mass stellar companions on late stellar evolution
• Gaia Early Data Release 3: Updated radial velocities from Gaia DR2
• Kepler-1661 b: A Neptune-sized Kepler Transiting Circumbinary Planet around a Grazing Eclipsing Binary
• Gaia Data Release 2: All-sky classification of high-amplitude pulsating stars
• An Ultra Metal-poor Star Near the Hydrogen-burning Limit
• Evidence that the Hot Jupiter WASP-77 A b Formed Beyond Its Parent Protoplanetary Disk's H2O Ice Line
• Kinematics and dynamics of Gaia red clump stars
• A stellar relic filament in the Orion star forming region
• The Milky Way's Shell Structure Reveals the Time of a Radial Collision
• The Next Generation Virgo Cluster Survey. XXXIV. Ultra-Compact Dwarf (UCD) Galaxies in the Virgo Cluster
• The $\mu$ Tau Association: A 60 Myr-Old Coeval Group at 150 pc from the Sun
• Lifetime of short-period binaries measured from their Galactic kinematics
• Completeness of the Gaia-verse I: when and where were Gaia's eyes on the sky during DR2?
• Volans-Carina: A New 90 Myr Old Stellar Association at 85 pc
• Gaia Data Release 2. Calibration and mitigation of electronic offset effects in the data
• Characterizing K2 Candidate Planetary Systems Orbiting Low-Mass Stars IV: Updated Properties for 86 Cool Dwarfs Observed During Campaigns 1-17
• Unveiling the star formation history of the Upper Scorpius association through its kinematics
• Sub-percent Photometry: Faint DA White Dwarf Spectophotometric Standards for Astrophysical Observatories
• Ultra-short-period Planets are Stable Against Tidal Inspiral
• Globular cluster candidates in the Galactic bulge: Gaia and VVV view of the latest discoveries
• A Reanalysis of the Fundamental Parameters and Age of TRAPPIST-1
• The Gaia-ESO Survey: detection and characterization of single line spectroscopic binaries
• Completeness of the Gaia-verse IV: The Astrometry Spread Function of Gaia DR2
• Revised Exoplanet Radii and Habitability Using Gaia Data Release 2
• Membership lists for 431 open clusters in Gaia DR2 using extreme deconvolution gaussian mixture models
• Astrophysical Parameters from Gaia DR2, 2MASS & AllWISE
• A Swing of the Pendulum: The Chemodynamics of the Local Stellar Halo Indicate Contributions from Several Radial Merger Events
• Photometric Cross-Calibration of the SDSS Stripe 82 Standard Stars catalogue with Gaia EDR3, and Comparison with Pan-STARRS1, DES, CFIS and GALEX catalogues
• Gaia Data Release 3: spectroscopic binary-star orbital solutions. The SB1 processing chain
• Correction to the photometric magnitudes of the Gaia Early Data Release 3
• Correction to the photometric colors of the Gaia Data Release 2 with the stellar color regression method
• Towards a more complete sample of binary central stars of planetary nebulae with Gaia
• K2-288Bb: A Small Temperate Planet in a Low-mass Binary System Discovered by Citizen Scientists
• Confirming NGC 6231 as the parent cluster of the runaway high-mass X-ray binary HD 153919/4U 1700-37 with Gaia DR2
• Characterising the Gaia Radial Velocity sample selection function in its native photometry
• Completeness of the Gaia-verse III: using hidden states to infer gaps, detection efficiencies and the scanning law from the DR2 light curves
• An Amusing Look at the Host of the Periodic Nuclear Transient ASASSN-14ko Reveals a Second AGN
• Planetary parameters, XUV environments and mass-loss rates for nearby gaseous planets with X-ray detected host-stars
• A Mini-Neptune Orbiting the Metal-poor K Dwarf BD+29 2654
• Optical properties of amorphous carbon dust around C-stars: new constraints from 2MASS and Gaia observations
• Extended envelopes around Galactic Cepheids. V. Multi-wavelength and time-dependent analysis of IR excess
• Assessing Spectroscopic-Binary Multiplicity Properties Using Robo-AO Imaging
• Gaia predicted brown dwarf detection rates around FGK stars in astrometry, radial velocity, and photometric transits
• The first heavy-metal hot subdwarf composite binary SB 744
• An absolute mass, precise age, and hints of planetary winds for WASP-121 A and b from a JWST NIRSpec phase curve
• Orbital and atmospheric parameters of two wide O-type subdwarf binaries: BD$-$11${}^{\mathrm{o}}$162 and Feige 80
• An Intriguing Globular Cluster in the Galactic Bulge from the VVV Survey
• TIC 278956474: Two close binaries in one young quadruple system, identified by \textit{TESS}
• NSV 1440: First WZ Sge-type Object in AM CVn stars and candidates
• The Gaia/IPHAS and Gaia/KIS Value-Added Catalogues
• A Search for Variable Stars in the Globular Cluster M4 with K2
• Potential asteroid discoveries by the ESA Gaia mission: Results from follow-up observations
• Optical night sky brightness measurements from the stratosphere
• Ultralow-amplitude RR Lyrae Stars in M4
• Reaching the boundary between stellar kinematic groups and very wide binaries. IV. The widest Washington double star systems with rho > 1000 arcsec in Gaia DR3
• Refractory and Volatile Species in the UV-to-IR Transmission Spectrum of Ultra-hot Jupiter WASP-178b with HST and JWST
• The Dragonfly Wide Field Survey. II. Accurate Total Luminosities and Colors of Nearby Massive Galaxies and Implications for the Galaxy Stellar Mass Function
• Identifying Candidate Optical Variables Using Gaia Data Release 2
• Properties of F Stars with Stable Radial Velocity Timeseries: A Useful Metric for Selecting Low-jitter F Stars
• Li-rich giant stars under scrutiny: Binarity, magnetic activity and the evolutionary status after Gaia DR2
• Simultaneous X-ray and optical variability of M dwarfs observed with eROSITA and TESS
• Gaia Focused Product Release: A catalogue of sources around quasars to search for strongly lensed quasars
• Multiple Stellar Populations outside the tidal radius of NGC1851 through Gaia DR3 XP Spectra
• Discovery of an M-type Companion to the Herbig Ae Star V1787 Ori
• Third Nearest WZ Sge-Type Dwarf Nova candidate ASASSN-14dx Classified on the Basis of Gaia Data Release 2
• Determining the Scale Length and Height of the Milky Way's Thick Disc Using RR~Lyrae
• A Search for Transiting Planets in the Globular Cluster M4 with K2: Candidates and Occurrence Limits
• An in-depth analysis of the differentially expanding star cluster Stock 18 (Villafranca O-036) using Gaia DR3 and ground-based data
• SB 796: a high-velocity RRc star
• XGAPS: a sub-arcsecond cross-match of Galactic Plane Surveys
• The Occurrence-weighted Median Planets Discovered by Transit Surveys Orbiting Solar-type Stars and Their Implications for Planet Formation and Evolution