Organised Randoms: Learning and correcting for systematic galaxy clustering patterns in KiDS using self-organising maps

Abstract

We present a new method for the mitigation of observational systematic effects in angular galaxy clustering via corrective random galaxy catalogues. Real and synthetic galaxy data, from the Kilo Degree Survey's (KiDS) 4${}^{\mathrm{th}}$ Data Release (KiDS-$1000$) and the Full-sky Lognormal Astro-fields Simulation Kit (FLASK) package respectively, are used to train self-organising maps (SOMs) to learn the multivariate relationships between observed galaxy number density and up to six systematic-tracer variables, including seeing, Galactic dust extinction, and Galactic stellar density. We then create `organised' randoms, i.e. random galaxy catalogues with spatially variable number densities, mimicking the learnt systematic density modes in the data. Using realistically biased mock data, we show that these organised randoms consistently subtract spurious density modes from the two-point angular correlation function $w(\vartheta )$, correcting biases of up to $12\sigma$ in the mean clustering amplitude to as low as $0.1\sigma$, over a high signal-to-noise angular range of 7-100 arcmin. Their performance is also validated for angular clustering cross-correlations in a bright, flux-limited subset of KiDS-$1000$, comparing against an analogous sample constructed from highly-complete spectroscopic redshift data. Each organised random catalogue object is a `clone' carrying the properties of a real galaxy, and is distributed throughout the survey footprint according to the parent galaxy's position in systematics-space. Thus, sub-sample randoms are readily derived from a single master random catalogue via the same selection as applied to the real galaxies. Our method is expected to improve in performance with increased survey area, galaxy number density, and systematic contamination, making organised randoms extremely promising for current and future clustering analyses of faint samples.

arXiv comment19 pages (7 appendix pages), 12 figures (8 appendix figures), accepted to A&A

References in corpus (64)

• Maps of Dust IR Emission for Use in Estimation of Reddening and CMBR Foregrounds
• The Sloan Digital Sky Survey: Technical Summary
• Measuring Reddening with SDSS Stellar Spectra and Recalibrating SFD
• HEALPix -- a Framework for High Resolution Discretization, and Fast Analysis of Data Distributed on the Sphere
• Nine-Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Cosmological Parameter Results
• The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: cosmological analysis of the DR12 galaxy sample
• Stable clustering, the halo model and nonlinear cosmological power spectra
• Large Magellanic Cloud Cepheid Standards Provide a 1% Foundation for the Determination of the Hubble Constant and Stronger Evidence for Physics Beyond LambdaCDM
• The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: Baryon Acoustic Oscillations in the Data Release 10 and 11 galaxy samples
• Overview of the DESI Legacy Imaging Surveys
• Dark Energy Survey Year 1 Results: Cosmological Constraints from Galaxy Clustering and Weak Lensing
• Revising the Halofit Model for the Nonlinear Matter Power Spectrum
• Spectroscopic Target Selection for the Sloan Digital Sky Survey: The Luminous Red Galaxy Sample
• KiDS-450: Cosmological parameter constraints from tomographic weak gravitational lensing
• Why your model parameter confidences might be too optimistic -- unbiased estimation of the inverse covariance matrix
• KiDS-1000 Cosmology: Cosmic shear constraints and comparison between two point statistics
• KiDS-1000 Cosmology: Multi-probe weak gravitational lensing and spectroscopic galaxy clustering constraints
• Dark Energy Survey Year 1 Results: Cosmological Constraints from Cosmic Shear
• Gaia Data Release 2: Catalogue validation
• Galaxy And Mass Assembly (GAMA): end of survey report and data release 2
• Quasar-Lyman $\alpha$ Forest Cross-Correlation from BOSS DR11 : Baryon Acoustic Oscillations
• Extended Limber Approximation
• Gravitational Lensing Analysis of the Kilo Degree Survey
• The Kilo-Degree Survey
• CFHTLenS: The Canada-France-Hawaii Telescope Lensing Survey - Imaging Data and Catalogue Products
• GAMA: towards a physical understanding of galaxy formation
• The Skewness of the Aperture Mass Statistic
• The fourth data release of the Kilo-Degree Survey: ugri imaging and nine-band optical-IR photometry over 1000 square degrees
• KiDS-450 + 2dFLenS: Cosmological parameter constraints from weak gravitational lensing tomography and overlapping redshift-space galaxy clustering
• KiDS+GAMA: Cosmology constraints from a joint analysis of cosmic shear, galaxy-galaxy lensing and angular clustering
• ANNz2 - photometric redshift and probability distribution function estimation using machine learning
• Ameliorating Systematic Uncertainties in the Angular Clustering of Galaxies: A Study using SDSS-III
• Cosmology from large-scale structure: Constraining $\Lambda$CDM with BOSS
• KiDS+VIKING-450 and DES-Y1 combined: Cosmology with cosmic shear
• Improving lognormal models for cosmological fields
• KiDS-1000 catalogue: Redshift distributions and their calibration
• KiDS-1000 catalogue: Weak gravitational lensing shear measurements
• Clustering of Sloan Digital Sky Survey III Photometric Luminous Galaxies: The Measurement, Systematics and Cosmological Implications
• Dark Energy Survey Year 1 Results: Galaxy clustering for combined probes
• The Complete Calibration of the Color-Redshift Relation (C3R2) Survey: Survey Overview and Data Release 1
• Galaxy-galaxy lensing estimators and their covariance properties
• The Complete Calibration of the Color-Redshift Relation (C3R2) Survey: Analysis and Data Release 2
• Photometric Redshift Calibration with Self Organising Maps
• Phenotypic redshifts with self-organizing maps: A novel method to characterize redshift distributions of source galaxies for weak lensing
• No galaxy left behind: accurate measurements with the faintest objects in the Dark Energy Survey
• The SDSS-IV Extended Baryon Oscillation Spectroscopic Survey: final Emission Line Galaxy Target Selection
• Estimating the large-scale angular power spectrum in the presence of systematics: a case study of Sloan Digital Sky Survey quasars
• Unsupervised self-organised mapping: a versatile empirical tool for object selection, classification and redshift estimation in large surveys
• Dark Energy Survey Year 3 Results: Optimizing the Lens Sample in Combined Galaxy Clustering and Galaxy-Galaxy Lensing Analysis
• Photometric redshifts for the Kilo-Degree Survey. Machine-learning analysis with artificial neural networks
• Galaxy and Mass Assembly (GAMA): Projected Galaxy Clustering
• Tomographic galaxy clustering with the Subaru Hyper Suprime-Cam first year public data release
• Exploiting the full potential of photometric quasar surveys: Optimal power spectra through blind mitigation of systematics
• Improving Galaxy Clustering Measurements with Deep Learning: analysis of the DECaLS DR7 data
• Mitigating contamination in LSS surveys: a comparison of methods
• Blinding multi-probe cosmological experiments
• Horizon-AGN virtual observatory -- 2: Template-free estimates of galaxy properties from colours
• Imaging Systematics and Clustering of DESI Main Targets
• Mitigating Systematic Errors in Angular Correlation Function Measurements from Wide Field Surveys
• Constraints on Cosmology and Baryonic Feedback with the Deep Lens Survey Using Galaxy-Galaxy and Galaxy-Mass Power Spectra
• Blind Observers of the Sky
• Minimising the impact of scale-dependent galaxy bias on the joint cosmological analysis of large scale structures
• A blinding solution for inference from astronomical data
• Linear Systematics Mitigation in Galaxy Clustering in the Dark Energy Survey Year 1 Data

Cited by in corpus (18)

• Euclid. I. Overview of the Euclid mission
• Dark Energy Survey Year 3 Results: Galaxy clustering and systematics treatment for lens galaxy samples
• Dark Energy Survey Year 3 Results: Measuring the Survey Transfer Function with Balrog
• Bright galaxy sample in the Kilo-Degree Survey Data Release 4: selection, photometric redshifts, and physical properties
• KiDS-1000: Combined halo-model cosmology constraints from galaxy abundance, galaxy clustering and galaxy-galaxy lensing
• KiDS-SBI: Simulation-based inference analysis of KiDS-1000 cosmic shear
• KiDS-1000: cross-correlation with Planck cosmic microwave background lensing and intrinsic alignment removal with self-calibration
• Clustering of red-sequence galaxies in the fourth data release ofthe Kilo-Degree Survey
• KiDS-Legacy: Covariance validation and the unified OneCovariance framework for projected large-scale structure observables
• Euclid preparation. LIX. Angular power spectra from discrete observations
• KiDS-Legacy: Angular galaxy clustering from deep surveys with complex selection effects
• YOUNG Star detrending for Transiting Exoplanet Recovery (YOUNGSTER) II: Using Self-Organising Maps to explore young star variability in Sectors 1-13 of TESS data
• Joint inference of multiplicative and additive systematics in galaxy density fluctuations and clustering measurements
• Euclid: Photometric redshift calibration with self-organising maps
• Spurious correlations between galaxies and multi-epoch image stacks in the DESI Legacy Surveys
• 6x2pt: Forecasting gains from joint weak lensing and galaxy clustering analyses with spectroscopic-photometric galaxy cross-correlations
• Impact of Large-Scale Structure Systematics on Cosmological Parameter Estimation
• Systematics mitigation for catalogue-based angular power spectra