MLGWSC-1: The first Machine Learning Gravitational-Wave Search Mock Data Challenge

Abstract

We present the results of the first Machine Learning Gravitational-Wave Search Mock Data Challenge (MLGWSC-1). For this challenge, participating groups had to identify gravitational-wave signals from binary black hole mergers of increasing complexity and duration embedded in progressively more realistic noise. The final of the 4 provided datasets contained real noise from the O3a observing run and signals up to a duration of 20 seconds with the inclusion of precession effects and higher order modes. We present the average sensitivity distance and runtime for the 6 entered algorithms derived from 1 month of test data unknown to the participants prior to submission. Of these, 4 are machine learning algorithms. We find that the best machine learning based algorithms are able to achieve up to 95% of the sensitive distance of matched-filtering based production analyses for simulated Gaussian noise at a false-alarm rate (FAR) of one per month. In contrast, for real noise, the leading machine learning search achieved 70%. For higher FARs the differences in sensitive distance shrink to the point where select machine learning submissions outperform traditional search algorithms at FARs $\ge 200$ per month on some datasets. Our results show that current machine learning search algorithms may already be sensitive enough in limited parameter regions to be useful for some production settings. To improve the state-of-the-art, machine learning algorithms need to reduce the false-alarm rates at which they are capable of detecting signals and extend their validity to regions of parameter space where modeled searches are computationally expensive to run. Based on our findings we compile a list of research areas that we believe are the most important to elevate machine learning searches to an invaluable tool in gravitational-wave signal detection.

25 pages, 6 figures, 4 tables, additional material available at https://github.com/gwastro/ml-mock-data-challenge-1

References in corpus (41)

• GW170817: Observation of Gravitational Waves from a Binary Neutron Star Inspiral
• Advanced Virgo: a 2nd generation interferometric gravitational wave detector
• Advanced LIGO
• GWTC-2: Compact Binary Coalescences Observed by LIGO and Virgo During the First Half of the Third Observing Run
• An improved effective-one-body model of spinning, nonprecessing binary black holes for the era of gravitational-wave astrophysics with advanced detectors
• Towards models of gravitational waveforms from generic binaries II: Modelling precession effects with a single effective precession parameter
• Gravity Spy: Integrating Advanced LIGO Detector Characterization, Machine Learning, and Citizen Science
• Real-time gravitational-wave science with neural posterior estimation
• 3-OGC: Catalog of gravitational waves from compact-binary mergers
• Search for gravitational waves from binary inspirals in S3 and S4 LIGO data
• Nested Sampling with Normalising Flows for Gravitational-Wave Inference
• Gravitational waves from inspiralling compact binaries: hexagonal template placement and its efficiency in detecting physical signals
• Review of the Advanced LIGO gravitational wave observatories leading to observing run four
• Applications and Techniques for Fast Machine Learning in Science
• Accelerated, Scalable and Reproducible AI-driven Gravitational Wave Detection
• Detection of gravitational-wave signals from binary neutron star mergers using machine learning
• Broad search for gravitational waves from subsolar-mass binaries through LIGO and Virgo's third observing run
• Detection and Parameter Estimation of Gravitational Waves from Binary Neutron-Star Mergers in Real LIGO Data using Deep Learning
• Deep Learning Ensemble for Real-time Gravitational Wave Detection of Spinning Binary Black Hole Mergers
• Deep-Learning Continuous Gravitational Waves: Multiple detectors and realistic noise
• Deep learning for gravitational wave forecasting of neutron star mergers
• Training Strategies for Deep Learning Gravitational-Wave Searches
• Using Deep Learning to Localize Gravitational Wave Sources
• Denoising Gravitational Waves with Enhanced Deep Recurrent Denoising Auto-Encoders
• From One to Many: A Deep Learning Coincident Gravitational-Wave Search
• First gravitational-wave search for intermediate-mass black hole mergers with higher order harmonics
• Anomaly Detection in Gravitational Waves data using Convolutional AutoEncoders
• Optimization of model independent gravitational wave search using machine learning
• Inference-optimized AI and high performance computing for gravitational wave detection at scale
• Deep Learning with Quantized Neural Networks for Gravitational Wave Forecasting of Eccentric Compact Binary Coalescence
• Improving significance of binary black hole mergers in Advanced LIGO data using deep learning : Confirmation of GW151216
• GWSkyNet: a real-time classifier for public gravitational-wave candidates
• Deep learning for clustering of continuous gravitational wave candidates II: identification of low-SNR candidates
• Utilizing Gaussian mixture models in all-sky searches for short-duration gravitational wave bursts
• Search for binary black hole mergers in the third observing run of Advanced LIGO-Virgo using coherent WaveBurst enhanced with machine learning
• Convolutional neural network for gravitational-wave early alert: Going down in frequency
• Hierarchical approach to matched filtering using a reduced basis
• Incorporating information from LIGO data quality streams into the PyCBC search for gravitational waves
• Assessing the impact of non-Gaussian noise on convolutional neural networks that search for continuous gravitational waves
• Sensitivity of spin-aligned searches for neutron star-black hole systems using future detectors
• AI and extreme scale computing to learn and infer the physics of higher order gravitational wave modes of quasi-circular, spinning, non-precessing binary black hole mergers

Cited by in corpus (33)

• The Science of the Einstein Telescope
• Deep Residual Networks for Gravitational Wave Detection
• Deep Learning Detection and Classification of Gravitational Waves from Neutron Star-Black Hole Mergers
• New Gravitational Wave Discoveries Enabled by Machine Learning
• Applications of machine learning in gravitational wave research with current interferometric detectors
• Numerical relativity higher order gravitational waveforms of eccentric, spinning, non-precessing binary black hole mergers
• Searches for Mass-Asymmetric Compact Binary Coalescence Events using Neural Networks in the LIGO/Virgo Third Observation Period
• Premerger detection of massive black hole binaries using deep learning
• Comparative study of 1D and 2D convolutional neural network models with attribution analysis for gravitational wave detection from compact binary coalescences
• Neural network time-series classifiers for gravitational-wave searches in single-detector periods
• Decoding Long-duration Gravitational Waves from Binary Neutron Stars with Machine Learning: Parameter Estimation and Equations of State
• Parameter estimation of microlensed gravitational waves with Conditional Variational Autoencoders
• Rapid identification of time-frequency domain gravitational wave signals from binary black holes using deep learning
• Searching for gravitational waves from stellar-mass binary black holes early inspiral
• Convolutional Neural Networks for signal detection in real LIGO data
• Novel neural-network architecture for continuous gravitational waves
• Deep-Learning Classification and Parameter Inference of Rotational Core-Collapse Supernovae
• Binary Neutron Star Merger Search Pipeline Powered by Deep Learning
• Reconstruction of binary black hole harmonics in LIGO using deep learning
• New approach to template banks of gravitational waves with higher harmonics: Reducing matched-filtering cost by over an order of magnitude
• Navigating Unknowns: Deep Learning Robustness for Gravitational Wave Signal Reconstruction
• Novel Deep Learning Approach to Detecting Binary Black Hole Mergers
• Physics-inspired spatiotemporal-graph AI ensemble for the detection of higher order wave mode signals of spinning binary black hole mergers
• Identifying and Mitigating Machine Learning Biases for the Gravitational Wave Detection Problem
• Comparison of neural network architectures for feature extraction from binary black hole merger waveforms
• Beyond FINDCHIRP: Breaking the memory wall and optimal FFTs for Gravitational-Wave Matched-Filter Searches with Ratio-Filter Dechirping
• Search for exotic gravitational wave signals beyond general relativity using deep learning
• Learning to detect continuous gravitational waves: an open data-analysis competition
• Learning and Interpreting Gravitational-Wave Features from CNNs with a Random Forest Approach
• Machine Learning Confirms GW231123 is a "Lite" Intermediate Mass Black Hole Merger
• Towards an anomaly detection pipeline for gravitational waves at the Einstein Telescope
• Parameter inference of millilensed gravitational waves using neural spline flows
• Unveiling gravitational waves from core-collapse supernovae with MUSE