Applications and Techniques for Fast Machine Learning in Science

Abstract

In this community review report, we discuss applications and techniques for fast machine learning (ML) in science -- the concept of integrating power ML methods into the real-time experimental data processing loop to accelerate scientific discovery. The material for the report builds on two workshops held by the Fast ML for Science community and covers three main areas: applications for fast ML across a number of scientific domains; techniques for training and implementing performant and resource-efficient ML algorithms; and computing architectures, platforms, and technologies for deploying these algorithms. We also present overlapping challenges across the multiple scientific domains where common solutions can be found. This community report is intended to give plenty of examples and inspiration for scientific discovery through integrated and accelerated ML solutions. This is followed by a high-level overview and organization of technical advances, including an abundance of pointers to source material, which can enable these breakthroughs.

arXiv comment66 pages, 13 figures, 5 tables

References in corpus (206)

• Scikit-learn: Machine Learning in Python
• XGBoost: A Scalable Tree Boosting System
• Observational Evidence from Supernovae for an Accelerating Universe and a Cosmological Constant
• Measurements of Omega and Lambda from 42 High-Redshift Supernovae
• Observation of Gravitational Waves from a Binary Black Hole Merger
• Distilling the Knowledge in a Neural Network
• First Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Determination of Cosmological Parameters
• Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC
• Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• GW170817: Observation of Gravitational Waves from a Binary Neutron Star Inspiral
• Seven-Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Cosmological Interpretation
• Detection of the Baryon Acoustic Peak in the Large-Scale Correlation Function of SDSS Luminous Red Galaxies
• Advanced Virgo: a 2nd generation interferometric gravitational wave detector
• Supersymmetric Dark Matter
• Neural Architecture Search with Reinforcement Learning
• Multi-messenger Observations of a Binary Neutron Star Merger
• LSST: from Science Drivers to Reference Design and Anticipated Data Products
• Deep Learning with Coherent Nanophotonic Circuits
• The Future of Digital Health with Federated Learning
• The Complete Light-curve Sample of Spectroscopically Confirmed Type Ia Supernovae from Pan-STARRS1 and Cosmological Constraints from The Combined Pantheon Sample
• All-Optical Machine Learning Using Diffractive Deep Neural Networks
• Machine learning and the physical sciences
• Recent Advances in Physical Reservoir Computing: A Review
• FitNets: Hints for Thin Deep Nets
• Improved cosmological constraints from a joint analysis of the SDSS-II and SNLS supernova samples
• Baryon Acoustic Oscillations in the Sloan Digital Sky Survey Data Release 7 Galaxy Sample
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• The Cosmic Microwave Background Spectrum from the Full COBE/FIRAS Data Set
• Photonics for artificial intelligence and neuromorphic computing
• Axions In String Theory
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
• Deep Learning in Spiking Neural Networks
• DARTS: Differentiable Architecture Search
• AMC: AutoML for Model Compression and Acceleration on Mobile Devices
• Multi-messenger observations of a flaring blazar coincident with high-energy neutrino IceCube-170922A
• Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition
• Interferometer design of the KAGRA gravitational wave detector
• FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
• Properties of the Binary Black Hole Merger GW150914
• Particle-flow reconstruction and global event description with the CMS detector
• 21-cm cosmology
• Data-driven discovery of coordinates and governing equations
• Learning to Optimize: Training Deep Neural Networks for Wireless Resource Management
• Baryon Acoustic Oscillations in the Lyα forest of BOSS DR11 quasars
• The T2K Experiment
• Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
• Trained Ternary Quantization
• Planck 2015 results. XI. CMB power spectra, likelihoods, and robustness of parameters
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Scalar Dark Matter candidates
• Probing condensed matter physics with magnetometry based on nitrogen-vacancy centres in diamond
• On the Variance of the Adaptive Learning Rate and Beyond
• Detection of Weak Gravitational Lensing by Large-scale Structure
• Vowel recognition with four coupled spin-torque nano-oscillators
• SNIP: Single-shot Network Pruning based on Connection Sensitivity
• Final model independent result of DAMA/LIBRA-phase1
• High-Precision Tuning of State for Memristive Devices by Adaptable Variation-Tolerant Algorithm
• Cold dark matter: controversies on small scales
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Direct Detection of WIMP Dark Matter: Concepts and Status
• Accurate deep neural network inference using computational phase-change memory
• The State of Sparsity in Deep Neural Networks
• Fast inference of deep neural networks in FPGAs for particle physics
• Tuning the conductance of single-walled carbon nanotubes by ion irradiation in the Anderson localization regime
• Candidate Electromagnetic Counterpart to the Binary Black Hole Merger Gravitational Wave Event S190521g
• Large-Scale Optical Neural Networks based on Photoelectric Multiplication
• Improving galaxy morphologies for SDSS with Deep Learning
• CaloGAN: Simulating 3D High Energy Particle Showers in Multi-Layer Electromagnetic Calorimeters with Generative Adversarial Networks
• Deep Learning for Real-time Gravitational Wave Detection and Parameter Estimation: Results with Advanced LIGO Data
• Lookahead Optimizer: k steps forward, 1 step back
• The Strong CP Problem and Axions
• H0LiCOW I. $H_0$ Lenses in COSMOGRAIL's Wellspring: Program Overview
• The morphology of HII regions during reionization
• Microscope 2.0: An Augmented Reality Microscope with Real-time Artificial Intelligence Integration
• Efficient, reliable and fast high-level triggering using a bonsai boosted decision tree
• Gravity Spy: Integrating Advanced LIGO Detector Characterization, Machine Learning, and Citizen Science
• Supernova Neutrino Detection
• A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis
• Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups
• Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis
• Projected Sensitivity of the SuperCDMS SNOLAB experiment
• Dynamic Graph CNN for Learning on Point Clouds
• Accelerating Science with Generative Adversarial Networks: An Application to 3D Particle Showers in Multi-Layer Calorimeters
• SINDy-PI: A Robust Algorithm for Parallel Implicit Sparse Identification of Nonlinear Dynamics
• First Cosmology Results using Type Ia Supernovae from the Dark Energy Survey: Constraints on Cosmological Parameters
• Model compression via distillation and quantization
• Matching matched filtering with deep networks in gravitational-wave astronomy
• SNEWS: The SuperNova Early Warning System
• Data-Free Knowledge Distillation for Deep Neural Networks
• Bayesian parameter estimation using conditional variational autoencoders for gravitational-wave astronomy
• The Automatic Learning for the Rapid Classification of Events (ALeRCE) Alert Broker
• Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors
• Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search
• Dark Energy Survey Year 1 Results: Measurement of the Baryon Acoustic Oscillation scale in the distribution of galaxies to redshift 1
• SuperNNova: an open-source framework for Bayesian, Neural Network based supernova classification
• DeepSphere: Efficient spherical Convolutional Neural Network with HEALPix sampling for cosmological applications
• First Season MWA EoR Power Spectrum Results at Redshift 7
• Enhancing Gravitational-Wave Science with Machine Learning
• New constraints on oscillation parameters from ${\nu }_e$ appearance and ${\nu }_{\mu }$ disappearance in the NOvA experiment
• Loss-aware Binarization of Deep Networks
• Science Requirements and Detector Concepts for the Electron-Ion Collider: EIC Yellow Report
• Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy
• Improving Neural Network Quantization without Retraining using Outlier Channel Splitting
• Neural Networks with Few Multiplications
• Convolutional neural networks: a magic bullet for gravitational-wave detection?
• Constraints on Oscillation Parameters from ${\nu }_e$ Appearance and ${\nu }_{\mu }$ Disappearance in NOvA
• Joint Cosmic Shear Measurements with the Keck and William Herschel Telescopes
• Machine Learning for the Zwicky Transient Facility
• Cosmological Constraints from the double source plane lens SDSSJ0946+1006
• Learning Bayesian posteriors with neural networks for gravitational-wave inference
• Learning representations of irregular particle-detector geometry with distance-weighted graph networks
• Machine-learning non-stationary noise out of gravitational wave detectors
• Fast convolutional neural networks on FPGAs with hls4ml
• Nanoscale electrical conductivity imaging using a nitrogen-vacancy center in diamond
• Supernova Neutrino Burst Detection with the Deep Underground Neutrino Experiment
• Post-training 4-bit quantization of convolution networks for rapid-deployment
• Application of machine learning algorithms to the study of noise artifacts in gravitational-wave data
• Compressing deep neural networks on FPGAs to binary and ternary precision with HLS4ML
• Quantum neuromorphic computing
• Fink, a new generation of broker for the LSST community
• Calorimetry with Deep Learning: Particle Simulation and Reconstruction for Collider Physics
• SNEWS 2.0: A Next-Generation SuperNova Early Warning System for Multi-messenger Astronomy
• Machine Learning-based Brokers for Real-time Classification of the LSST Alert Stream
• Image-based deep learning for classification of noise transients in gravitational wave detectors
• Fast cosmic web simulations with generative adversarial networks
• Fast and accurate simulation of particle detectors using generative adversarial networks
• Moonshine: Distilling with Cheap Convolutions
• DijetGAN: A Generative-Adversarial Network Approach for the Simulation of QCD Dijet Events at the LHC
• Deep-Learning continuous gravitational waves
• Deep Learning Identifies High-z Galaxies in a Central Blue Nugget Phase in a Characteristic Mass Range
• Graph Neural Networks for Particle Reconstruction in High Energy Physics detectors
• Application of Quantum Machine Learning using the Quantum Kernel Algorithm on High Energy Physics Analysis at the LHC
• Noise Reduction in Gravitational-wave Data via Deep Learning
• Zero-Shot Knowledge Distillation in Deep Networks
• All-silicon light-emitting diodes waveguide-integrated with superconducting single-photon detectors
• Stochastic Gradient Methods with Layer-wise Adaptive Moments for Training of Deep Networks
• Model-Independent Comparison of Direct vs. Indirect Detection of Supersymmetric Dark Matter
• Synchronization dynamics on the picosecond timescale in coupled Josephson junction neurons
• Fast inference of Boosted Decision Trees in FPGAs for particle physics
• Detection and Classification of Supernova Gravitational Waves Signals: A Deep Learning Approach
• A New Method to Observe Gravitational Waves emitted by Core Collapse Supernovae
• MLPF: Efficient machine-learned particle-flow reconstruction using graph neural networks
• Coherent Ising machines -- Quantum optics and neural network perspectives
• FPGA-accelerated machine learning inference as a service for particle physics computing
• Machine Learning and Cosmological Simulations II: Hydrodynamical Simulations
• Tomography of Massive Stars from Core Collapse to Supernova Shock Breakout
• Deep learning for clustering of continuous gravitational wave candidates
• A Compiler Infrastructure for Accelerator Generators
• Charged particle tracking via edge-classifying interaction networks
• Shampoo: Preconditioned Stochastic Tensor Optimization
• Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference
• Euclid Mission: building of a Reference Survey
• A Review of Basic Energy Reconstruction Techniques in Liquid Xenon and Argon Detectors for Dark Matter and Neutrino Physics Using NEST
• Distance-Weighted Graph Neural Networks on FPGAs for Real-Time Particle Reconstruction in High Energy Physics
• Full-sky Cosmic Microwave Background Foreground Cleaning Using Machine Learning
• Machine Learning and Cosmological Simulations I: Semi-Analytical Models
• Vertex and Energy Reconstruction in JUNO with Machine Learning Methods
• Graph Neural Networks for Particle Tracking and Reconstruction
• Machine Learning Analysis of Perovskite Oxides Grown by Molecular Beam Epitaxy
• Neutrino interaction classification with a convolutional neural network in the DUNE far detector
• Finding Symmetry Breaking Order Parameters with Euclidean Neural Networks
• Novel deep learning methods for track reconstruction
• A Convolutional Neural Network for Multiple Particle Identification in the MicroBooNE Liquid Argon Time Projection Chamber
• Convolutional Neural Networks for Direct Detection of Dark Matter
• PySINDy: A Python package for the Sparse Identification of Nonlinear Dynamics from Data
• Technical Design Report (TDR): Searching for a Sterile Neutrino at J-PARC MLF (E56, JSNS2)
• Efficient Monte Carlo Integration Using Boosted Decision Trees and Generative Deep Neural Networks
• LHC analysis-specific datasets with Generative Adversarial Networks
• Accelerated Charged Particle Tracking with Graph Neural Networks on FPGAs
• Exploring particle dynamics during self-organization processes via rotationally invariant latent representations
• PyHessian: Neural Networks Through the Lens of the Hessian
• Toward Decoding the Relationship between Domain Structure and Functionality in Ferroelectrics via Hidden Latent Variables
• EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis
• Streamlined Deployment for Quantized Neural Networks
• Cosmology with the SKA -- overview
• GPU-accelerated machine learning inference as a service for computing in neutrino experiments
• Accelerating Recurrent Neural Networks for Gravitational Wave Experiments
• Quantization for Rapid Deployment of Deep Neural Networks
• FPGAs-as-a-Service Toolkit (FaaST)
• Reducing model bias in a deep learning classifier using domain adversarial neural networks in the MINERvA experiment
• Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorization
• A Review on Machine Learning for Neutrino Experiments
• Application of Artificial Neural Network to Search for Gravitational-Wave Signals Associated with Short Gamma-Ray Bursts
• LadaBERT: Lightweight Adaptation of BERT through Hybrid Model Compression
• Implementation of high-performance, sub-microsecond deep neural networks on FPGAs for trigger applications
• HAWQV3: Dyadic Neural Network Quantization
• Combination of Domain Knowledge and Deep Learning for Sentiment Analysis
• I-BERT: Integer-only BERT Quantization
• The Mu2e experiment at Fermilab: a search for lepton flavor violation
• 4K-Memristor Analog-Grade Passive Crossbar Circuit
• Machine Learning Accelerated Likelihood-Free Event Reconstruction in Dark Matter Direct Detection
• Learning Discrepancy Models From Experimental Data
• Ranking Candidate Signals with Machine Learning in Low-Latency Search for Gravitational-Waves from Compact Binary Mergers
• Reservoir Computing with Superconducting Electronics
• Fast convolutional neural networks for identifying long-lived particles in a high-granularity calorimeter
• Scalable, End-to-End, Deep-Learning-Based Data Reconstruction Chain for Particle Imaging Detectors
• Cosmic Background Removal with Deep Neural Networks in SBND
• Data Augmentation at the LHC through Analysis-specific Fast Simulation with Deep Learning
• Hessian-Aware Pruning and Optimal Neural Implant
• Real-time Artificial Intelligence for Accelerator Control: A Study at the Fermilab Booster
• A Dynamic Reduction Network for Point Clouds
• GPU acceleration of a patient-specific airway image segmentation and its assessment
• Instance Segmentation GNNs for One-Shot Conformal Tracking at the LHC
• PAR: Personal Activity Radius Camera View for Contextual Sensing

Cited by in corpus (12)

• Machine Learning in Nuclear Physics
• MLGWSC-1: The first Machine Learning Gravitational-Wave Search Mock Data Challenge
• Deep Learning Detection and Classification of Gravitational Waves from Neutron Star-Black Hole Mergers
• Ephemeral Learning -- Augmenting Triggers with Online-Trained Normalizing Flows
• Machine learning light hypernuclei
• Localization of gravitational waves using machine learning
• Predicting Solid State Material Platforms for Quantum Technologies
• Neural network accelerator for quantum control
• Embedded FPGA Developments in 130nm and 28nm CMOS for Machine Learning in Particle Detector Readout
• Low latency optical-based mode tracking with machine learning deployed on FPGAs on a tokamak
• Development of a resource-efficient FPGA-based neural network regression model for the ATLAS muon trigger upgrades
• Variational Autoencoders for At-Source Data Reduction and Anomaly Detection in High Energy Particle Detectors