Galaxy merger challenge: A comparison study between machine learning-based detection methods

Abstract

Various galaxy merger detection methods have been applied to diverse datasets. However, it is difficult to understand how they compare. We aim to benchmark the relative performance of machine learning (ML) merger detection methods. We explore six leading ML methods using three main datasets. The first one (the training data) consists of mock observations from the IllustrisTNG simulations and allows us to quantify the performance metrics of the detection methods. The second one consists of mock observations from the Horizon-AGN simulations, introduced to evaluate the performance of classifiers trained on different, but comparable data. The third one consists of real observations from the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP) survey. For the binary classification task (mergers vs. non-mergers), all methods perform reasonably well in the domain of the training data. At $0.1<z<0.3$, precision and recall range between $\sim$70\% and 80\%, both of which decrease with increasing $z$ as expected (by $\sim$5\% for precision and $\sim$10\% for recall at $0.76<z<1.0$). When transferred to a different domain, the precision of all classifiers is only slightly reduced, but the recall is significantly worse (by $\sim$20-40\% depending on the method). Zoobot offers the best overall performance in terms of precision and F1 score. When applied to real HSC observations, all methods agree well with visual labels of clear mergers but can differ by more than an order of magnitude in predicting the overall fraction of major mergers. For the multi-class classification task to distinguish pre-, post- and non-mergers, none of the methods offer a good performance, which could be partly due to limitations in resolution and depth of the data. With the advent of better quality data (e.g. JWST and Euclid), it is important to improve our ability to detect mergers and distinguish between merger stages.

References in corpus (116)

• Deep Learning in Neural Networks: An Overview
• Planck 2015 results. XIII. Cosmological parameters
• Stellar population synthesis at the resolution of 2003
• Galactic Stellar and Substellar Initial Mass Function
• Seven-Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Cosmological Interpretation
• The Global Schmidt Law in Star Forming Galaxies
• Starburst99: Synthesis Models for Galaxies with Active Star Formation
• The EAGLE project: Simulating the evolution and assembly of galaxies and their environments
• Populating a cluster of galaxies - I. Results at z=0
• Evolutionary tracks and isochrones for low- and intermediate-mass stars: from 0.15 to 7 M_sun, and from Z=0.0004 to 0.03
• Introducing the Illustris Project: Simulating the coevolution of dark and visible matter in the Universe
• Cosmological Hydrodynamics with Adaptive Mesh Refinement: a new high resolution code called RAMSES
• Simulating Galaxy Formation with the IllustrisTNG Model
• First results from the IllustrisTNG simulations: matter and galaxy clustering
• First results from the IllustrisTNG simulations: the stellar mass content of groups and clusters of galaxies
• Accurate photometric redshifts for the CFHT Legacy Survey calibrated using the VIMOS VLT Deep Survey
• Radiative Transfer in a Clumpy Universe: II. The Utraviolet Extragalactic Background
• First results from the IllustrisTNG simulations: the galaxy color bimodality
• Galaxy Zoo : Morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey
• Properties of galaxies reproduced by a hydrodynamic simulation
• First results from the IllustrisTNG simulations: A tale of two elements -- chemical evolution of magnesium and europium
• Bayesian photometric redshift estimation
• FIRE-2 Simulations: Physics versus Numerics in Galaxy Formation
• The Hyper Suprime-Cam SSP Survey: Overview and Survey Design
• Substructures in hydrodynamical cluster simulations
• Dancing in the dark: galactic properties trace spin swings along the cosmic web
• A New Non-Parametric Approach to Galaxy Morphological Classification
• Rotation-invariant convolutional neural networks for galaxy morphology prediction
• The merger rate of galaxies in the Illustris Simulation: a comparison with observations and semi-empirical models
• Galaxy Zoo 2: detailed morphological classifications for 304,122 galaxies from the Sloan Digital Sky Survey
• Galaxy And Mass Assembly (GAMA): end of survey report and data release 2
• A Direct Measurement of Major Galaxy Mergers at z < 3
• The Hyper Suprime-Cam Software Pipeline
• The Horizon-AGN simulation: morphological diversity of galaxies promoted by AGN feedback
• The stellar mass assembly of galaxies in the Illustris simulation: growth by mergers and the spatial distribution of accreted stars
• The optical morphologies of galaxies in the IllustrisTNG simulation: a comparison to Pan-STARRS observations
• Improving galaxy morphologies for SDSS with Deep Learning
• Third Data Release of the Hyper Suprime-Cam Subaru Strategic Program
• The origin and implications of dark matter anisotropic cosmic infall on ~L* haloes
• The fourth data release of the Kilo-Degree Survey: ugri imaging and nine-band optical-IR photometry over 1000 square degrees
• A Library of Theoretical Ultraviolet Spectra of Massive, Hot Stars for Evolutionary Synthesis
• A catalog of visual-like morphologies in the 5 CANDELS fields using deep-learning
• Cold flows and the first quasars
• Galaxy pairs in the Sloan Digital Sky Survey - IX: Merger-induced AGN activity as traced by the Wide-field Infrared Survey Explorer
• Building Merger Trees from Cosmological N-body Simulations
• The Effect of Mass Ratio on the Morphology and Time-scales of Disc Galaxy Mergers
• The Nearly Universal Merger Rate of Dark Matter Haloes in Lambda-CDM Cosmology
• Thermal Stability of White Dwarfs Accreting Hydrogen-rich Matter and Progenitors of Type Ia Supernovae
• A calibration of the relation between the abundance of close galaxy pairs and the rate of galaxy mergers
• Galaxy Zoo: the fraction of merging galaxies in the SDSS and their morphologies
• The role of mergers and halo spin in shaping galaxy morphology
• Galaxy Zoo DECaLS: Detailed Visual Morphology Measurements from Volunteers and Deep Learning for 314,000 Galaxies
• Galaxy Interactions Trigger Rapid Black Hole Growth: an unprecedented view from the Hyper Suprime-Cam Survey
• A definitive merger-AGN connection at z~0 with CFIS: mergers have an excess of AGN and AGN hosts are more frequently disturbed
• A chronicle of galaxy mass assembly in the EAGLE simulation
• The VIMOS VLT Deep Survey :Evolution of the major merger rate since z~1 from spectroscopicaly confirmed galaxy pairs
• Observational constraints on the merger history of galaxies since $z\approx 6$: Probabilistic galaxy pair counts in the CANDELS fields
• Minor Galaxy Interactions: Star Formation Rates and Galaxy Properties
• Galaxy Zoo: Probabilistic Morphology through Bayesian CNNs and Active Learning
• Interacting galaxies on FIRE-2: The connection between enhanced star formation and interstellar gas content
• Using transfer learning to detect galaxy mergers
• The role of mergers in driving morphological transformation over cosmic time
• Effect of galaxy mergers on star formation rates
• Sussing Merger Trees: The Merger Trees Comparison Project
• KiDS+VIKING-450: A new combined optical & near-IR dataset for cosmology and astrophysics
• A consistent measure of the merger histories of massive galaxies using close-pair statistics I: Major mergers at $z<3.5$
• Optimising Automatic Morphological Classification of Galaxies with Machine Learning and Deep Learning using Dark Energy Survey Imaging
• Deep learning predictions of galaxy merger stage and the importance of observational realism
• Galaxy Zoo: Quantitative Visual Morphological Classifications for 48,000 galaxies from CANDELS
• Galaxy pairs in the Sloan digital sky survey - VII: The merger -- luminous infra-red galaxy connection
• The Canada-France Imaging Survey: First results from the u-band component
• The DAWES review 10: The impact of deep learning for the analysis of galaxy surveys
• The Pair Fraction of Massive Galaxies at 0 < z < 3
• The diminishing importance of major galaxy mergers at higher redshifts
• Massive Close Pairs Measure Rapid Galaxy Assembly in Mergers at High Redshift
• Galaxy Zoo: Morphological Classifications for 120,000 Galaxies in HST Legacy Imaging
• The three phases of galaxy formation
• Galaxy Merger Rates up to z $\sim$ 3 using a Bayesian Deep Learning Model $-$ A Major-Merger classifier using IllustrisTNG Simulation data
• Deep Learning Identifies High-z Galaxies in a Central Blue Nugget Phase in a Characteristic Mass Range
• Identifying Galaxy Mergers in Observations and Simulations with Deep Learning
• Galaxy morphological classification in deep-wide surveys via unsupervised machine learning
• Convolutional neural network identification of galaxy post-mergers in UNIONS using IllustrisTNG
• The MUSE Hubble Ultra Deep Field Survey. IX. Evolution of galaxy merger fraction since z~6
• Interacting galaxies in the IllustrisTNG simulations -- I: Triggered star formation in a cosmological context
• Galaxy mergers in EAGLE do not induce a significant amount of black hole growth yet do increase the rate of luminous AGN
• Preparing for low surface brightness science with the Vera C. Rubin Observatory: characterisation of tidal features from mock images
• The role of mergers and interactions in driving the evolution of dwarf galaxies over cosmic time
• Identification of Low Surface Brightness Tidal Features in Galaxies Using Convolutional Neural Networks
• No Assembly Required: Mergers are Mostly Irrelevant for the Growth of Low-mass Dwarf Galaxies
• The relationship between fine galaxy stellar morphology and star formation activity in cosmological simulations: a deep learning view
• Automated Distant Galaxy Merger Classifications from Space Telescope Images using the Illustris Simulation
• Galaxy mergers moulding the circum-galactic medium - I. The impact of a major merger
• Accurate Identification of Galaxy Mergers with Imaging
• Detecting outliers in astronomical images with deep generative networks
• The merger fraction of post-starburst galaxies in UNIONS
• Galaxy Formation as a Cosmological Tool. I: The Galaxy Merger History as a Measure of Cosmological Parameters
• Galaxy Evolution in all Five CANDELS Fields and IllustrisTNG: Morphological, Structural, and the Major Merger Evolution to $z\sim 3$
• The Structures of Distant Galaxies - IV: A New Empirical Measurement of the Time-Scale for Galaxy Mergers - Implications for the Merger History
• The ALHAMBRA survey: Accurate merger fractions by PDF analysis of photometric close pairs
• ERGO-ML I: Inferring the assembly histories of IllustrisTNG galaxies from integral observable properties via invertible neural networks
• Morphological signatures of mergers in the TNG50 simulation and the Kilo-Degree Survey: the merger fraction from dwarfs to Milky Way-like galaxies
• Star formation characteristics of CNN-identified post-mergers in the Ultraviolet Near Infrared Optical Northern Survey (UNIONS)
• Galaxy pairs in the Sloan Digital Sky Survey -- XIV. Galaxy mergers do not lie on the Fundamental Metallicity Relation
• Statistics of galaxy mergers: bridging the gap between theory and observation
• A scaling relation between merger rate of galaxies and their close pair count
• North Ecliptic Pole merging galaxy catalogue
• The spatially resolved star formation history of mergers: A comparative study of the LIRGs IC1623, NGC6090, NGC2623, and Mice
• Extremely massive disc galaxies in the nearby Universe form through gas-rich minor mergers
• Towards a consistent framework of comparing galaxy mergers in observations and simulations
• Identifying Galaxy Mergers in Simulated CEERS NIRCam Images using Random Forests
• Identification of tidal features in deep optical galaxy images with Convolutional Neural Networks
• AGN in post-mergers from the Ultraviolet Near Infrared Optical Northern Survey
• The signature of dissipation in the mass-size relation: are bulges simply spheroids wrapped in a disc?
• Galaxy And Mass Assembly (GAMA): Comparing Visually and Spectroscopically Identified Galaxy Merger Samples
• Relaxed blue ellipticals: accretion-driven stellar growth is a key evolutionary channel for low mass elliptical galaxies
• A multi-band AGN-SFG classifier for extragalactic radio surveys using machine learning

Cited by in corpus (9)

• Dust and Power: Unravelling the merger - active galactic nucleus connection in the second half of cosmic history
• GalaxyGenius: Mock galaxy image generator for various telescopes from hydrodynamical simulations
• The spatially-resolved effect of mergers on the stellar mass assembly of MaNGA galaxies
• AGN -- host galaxy photometric decomposition using a fast, accurate and precise deep learning approach
• Classifying merger stages with adaptive deep learning and cosmological hydrodynamical simulations
• Identifying Astrophysical Anomalies in 99.6 Million Cutouts from the Hubble Legacy Archive Using AnomalyMatch
• The major merger-active galactic nucleus connection up to the cosmic noon
• Merger fraction in galaxy groups and clusters at z < 0.2: A non-parametric morphological study with Subaru Hyper Suprime-Cam
• SE3D: Building a radiative transfer emulator to fit panchromatic resolved galaxy observations with 3D models of dust and stars