Surveying the reach and maturity of machine learning and artificial intelligence in astronomy

Abstract

Machine learning (automated processes that learn by example in order to classify, predict, discover or generate new data) and artificial intelligence (methods by which a computer makes decisions or discoveries that would usually require human intelligence) are now firmly established in astronomy. Every week, new applications of machine learning and artificial intelligence are added to a growing corpus of work. Random forests, support vector machines, and neural networks (artificial, deep, and convolutional) are now having a genuine impact for applications as diverse as discovering extrasolar planets, transient objects, quasars, and gravitationally-lensed systems, forecasting solar activity, and distinguishing between signals and instrumental effects in gravitational wave astronomy. This review surveys contemporary, published literature on machine learning and artificial intelligence in astronomy and astrophysics. Applications span seven main categories of activity: classification, regression, clustering, forecasting, generation, discovery, and the development of new scientific insight. These categories form the basis of a hierarchy of maturity, as the use of machine learning and artificial intelligence emerges, progresses or becomes established.

arXiv comment40 pages, accepted for publication in WIREs Data Mining and Knowledge Discovery (9 November 2019)

References in corpus (198)

• The Sloan Digital Sky Survey: Technical Summary
• Gaia Data Release 2. Summary of the contents and survey properties
• The Gaia mission
• The Seventh Data Release of the Sloan Digital Sky Survey
• LSST: from Science Drivers to Reference Design and Anticipated Data Products
• Methods for Interpreting and Understanding Deep Neural Networks
• LOFAR: The LOw-Frequency ARray
• The Galaxy Evolution Explorer: A Space Ultraviolet Survey Mission
• Gaia Data Release 1. Summary of the astrometric, photometric, and survey properties
• The Zwicky Transient Facility: System Overview, Performance, and First Results
• Galaxy Zoo : Morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey
• First Results from the Catalina Real-time Transient Survey
• The Fourth Data Release of the Sloan Digital Sky Survey
• The Dark Energy Survey: more than dark energy - an overview
• Cosmology and fundamental physics with the Euclid satellite
• Rotation-invariant convolutional neural networks for galaxy morphology prediction
• SkyMapper and the Southern Sky Survey
• ANNz: estimating photometric redshifts using artificial neural networks
• The Kilo-Degree Survey
• Science with ASKAP - the Australian Square Kilometre Array Pathfinder
• A Catalog of Northern Stars With Annual Proper Motions Larger Than 0.15 Seconds of Arc (LSPM catalog - North)
• Data Mining and Machine Learning in Astronomy
• Improving galaxy morphologies for SDSS with Deep Learning
• Deep Learning for Real-time Gravitational Wave Detection and Parameter Estimation: Results with Advanced LIGO Data
• Identifying Exoplanets with Deep Learning: A Five Planet Resonant Chain around Kepler-80 and an Eighth Planet around Kepler-90
• A First Catalog of Variable Stars Measured by the Asteroid Terrestrial-impact Last Alert System (ATLAS)
• Gravity Spy: Integrating Advanced LIGO Detector Characterization, Machine Learning, and Citizen Science
• The first and second data releases of the Kilo-Degree Survey
• Deep Neural Networks to Enable Real-time Multimessenger Astrophysics
• A catalog of visual-like morphologies in the 5 CANDELS fields using deep-learning
• TPZ : Photometric redshift PDFs and ancillary information by using prediction trees and random forests
• Spectral Classification of Quasars in the Sloan Digital Sky Survey: Eigenspectra; Redshift and Luminosity Effects
• PHAT: PHoto-z Accuracy Testing
• Science With The Australian Square Kilometre Array Pathfinder
• A Pair of Bootes: A New Milky Way Satellite
• The Fundamental Properties of Galaxies and a New Galaxy Classification System
• Star-galaxy Classification Using Deep Convolutional Neural Networks
• Mapping the Galaxy Color-Redshift Relation: Optimal Photometric Redshift Calibration Strategies for Cosmology Surveys
• Estimating photometric redshifts with artificial neural networks
• ANNz2 - photometric redshift and probability distribution function estimation using machine learning
• A new method for unveiling Open Clusters in Gaia: new nearby Open Clusters confirmed by DR2
• Galaxy Interactions Trigger Rapid Black Hole Growth: an unprecedented view from the Hyper Suprime-Cam Survey
• Probabilistic Random Forest: A machine learning algorithm for noisy datasets
• Solar Flare Prediction Model with Three Machine-Learning Algorithms Using Ultraviolet Brightening and Vector Magnetogram
• CMU DeepLens: Deep Learning For Automatic Image-based Galaxy-Galaxy Strong Lens Finding
• Dark Energy Survey Year 1 Results: Measurement of the Baryon Acoustic Oscillation scale in the distribution of galaxies to redshift 1
• Finding Strong Gravitational Lenses in the Kilo Degree Survey with Convolutional Neural Networks
• The Fornax Deep Survey with VST. I. The extended and diffuse stellar halo of NGC~1399 out to 192 kpc
• Forecasting Solar Flares Using Magnetogram-based Predictors and Machine Learning
• Three-dimensional interstellar dust reddening maps of the Galactic plane
• Eccentric, nonspinning, inspiral, Gaussian-process merger approximant for the detection and characterization of eccentric binary black hole mergers
• Galaxy Types in the Sloan Digital Sky Survey Using Supervised Artificial Neural Networks
• The Strong Gravitational Lens Finding Challenge
• Redshift distributions of galaxies in the DES Science Verification shear catalogue and implications for weak lensing
• Classifying Radio Galaxies with Convolutional Neural Network
• Machine Learning for the Zwicky Transient Facility
• Uncertainties in Parameters Estimated with Neural Networks: Application to Strong Gravitational Lensing
• Finding strong lenses in CFHTLS using convolutional neural networks
• FRB microstructure revealed by the real-time detection of FRB170827
• The weirdest SDSS galaxies: results from an outlier detection algorithm
• Searching for Exoplanets Using Artificial Intelligence
• Predicting Solar Flares Using SDO/HMI Vector Magnetic Data Product and Random Forest Algorithm
• A recurrent neural network for classification of unevenly sampled variable stars
• Photometric redshifts with the Multilayer Perceptron Neural Network: application to the HDF-S and SDSS
• Selection of radio pulsar candidates using artificial neural networks
• Estimating Photometric Redshifts Using Support Vector Machines
• SOMz: photometric redshift PDFs with self organizing maps and random atlas
• Application of machine learning algorithms to the study of noise artifacts in gravitational-wave data
• LinKS: Discovering galaxy-scale strong lenses in the Kilo-Degree Survey using Convolutional Neural Networks
• GPz: Non-stationary sparse Gaussian processes for heteroscedastic uncertainty estimation in photometric redshifts
• K2 Variable Catalogue II: Machine Learning Classification of Variable Stars and Eclipsing Binaries in K2 Fields 0-4
• A Population of Compact Elliptical Galaxies Detected with the Virtual Observatory
• A Morphological Classification Model to Identify Unresolved PanSTARRS1 Sources: Application in the ZTF Real-Time Pipeline
• Finding high-redshift strong lenses in DES using convolutional neural networks
• Neural computation as a tool for galaxy classification : methods and examples
• The NASA Astrophysics Data System: Overview
• Refining mass formulas for astrophysical applications: a Bayesian neural network approach
• Initial Data Release from the INT Photometric H-alpha Survey of the Northern Galactic Plane (IPHAS)
• Machine Learning-based Brokers for Real-time Classification of the LSST Alert Stream
• Fast cosmic web simulations with generative adversarial networks
• An automatic taxonomy of galaxy morphology using unsupervised machine learning
• Deep Learning Identifies High-z Galaxies in a Central Blue Nugget Phase in a Characteristic Mass Range
• Radio Galaxy Zoo: Compact and extended radio source classification with deep learning
• A New Tool for CME Arrival Time Prediction Using Machine Learning Algorithms: CAT-PUMA
• Photometric redshifts for Quasars in multi band Surveys
• The High Time Resolution Universe survey XIV: Discovery of 23 pulsars through GPU-accelerated reprocessing
• A hybrid supervised/unsupervised machine learning approach to solar flare prediction
• Applying Deep Learning to Fast Radio Burst Classification
• LensExtractor: A Convolutional Neural Network in Search of Strong Gravitational Lenses
• VDES J2325-5229 a z=2.7 gravitationally lensed quasar discovered using morphology independent supervised machine learning
• Photometric redshifts for the Kilo-Degree Survey. Machine-learning analysis with artificial neural networks
• METAPHOR: A machine learning based method for the probability density estimation of photometric redshifts
• The-wiZZ: Clustering redshift estimation for everyone
• MeerKAT Key Project Science, Specifications, and Proposals
• Painting galaxies into dark matter halos using machine learning
• Classification methods for noise transients in advanced gravitational-wave detectors II: performance tests on Advanced LIGO data
• Methodology study of machine learning for the neutron star equation of state
• New Galactic supernova remnants discovered with IPHAS
• Evolution of galaxy size--stellar mass relation from the Kilo Degree Survey
• Separation of pulsar signals from noise with supervised machine learning algorithms
• Carbon stars identified from LAMOST DR4 using Machine Learning
• Anomaly detection for machine learning redshifts applied to SDSS galaxies
• Integrating human and machine intelligence in galaxy morphology classification tasks
• Star-Galaxy Classification in Multi-Band Optical Imaging
• On the realistic validation of photometric redshifts, or why Teddy will never be Happy
• Machine learning cosmological structure formation
• Feature importance for machine learning redshifts applied to SDSS galaxies
• Modeling the Impact of Baryons on Subhalo Populations with Machine Learning
• Data Mining for Gravitationally Lensed Quasars
• Improved $\gamma$/hadron separation for the detection of faint gamma-ray sources using boosted decision trees
• A Machine Learning Based Morphological Classification of 14,245 Radio AGNs Selected From The Best-Heckman Sample
• Machine learning search for variable stars
• The Clustering of High-Redshift (2.9 $\le$ z $\le$ 5.1) Quasars in SDSS Stripe 82
• Using Machine Learning Methods to Forecast If Solar Flares Will Be Associated with CMEs and SEPs
• DeepStreaks: identifying fast-moving objects in the Zwicky Transient Facility data with deep learning
• DeepSource: Point Source Detection using Deep Learning
• Deblending galaxy superpositions with branched generative adversarial networks
• Discovering the Unexpected in Astronomical Survey Data
• Radio Galaxy Zoo: Machine learning for radio source host galaxy cross-identification
• Systematic Serendipity: A Test of Unsupervised Machine Learning as a Method for Anomaly Detection
• IPHAS discoveries of young stars towards Cyg OB2 and its southern periphery
• Finding flares in Kepler data using machine learning tools
• Transit Shapes and Self Organising Maps as a Tool for Ranking Planetary Candidates: Application to Kepler and K2
• Data Mining for Dwarf Novae in SDSS, GALEX and Astrometric Catalogues
• Using Oblique Decision Trees for the Morphological Classification of Galaxies
• Single-pulse classifier for the LOFAR Tied-Array All-sky Survey
• Effective Image Differencing with ConvNets for Real-time Transient Hunting
• Discovery of 36 eclipsing EL CVn binaries found by the Palomar Transient Factory
• Convolutional Neural Networks for Transient Candidate Vetting in Large-Scale Surveys
• Detecting outliers and learning complex structures with large spectroscopic surveys - a case study with APOGEE stars
• Forging new worlds: high-resolution synthetic galaxies with chained generative adversarial networks
• The Extremely Luminous Quasar Survey (ELQS) in the SDSS footprint I.: Infrared Based Candidate Selection
• A cooperative approach among methods for photometric redshifts estimation: an application to KiDS data
• Machine-assisted discovery of relationships in astronomy
• Evaluating the optical classification of Fermi BCUs using machine learning
• Supervised detection of exoplanets in high-contrast imaging sequences
• Photometric redshifts with Quasi Newton Algorithm (MLPQNA). Results in the PHAT1 contest
• Catalog of quasars from the Kilo-Degree Survey Data Release 3
• XMMPZCAT: A catalogue of photometric redshifts for X-ray sources
• Analysis of a Custom Support Vector Machine for Photometric Redshift Estimation and the Inclusion of Galaxy Shape Information
• Support Vector Machine classification of strong gravitational lenses
• Clustering the Orion B giant molecular cloud based on its molecular emission
• Astrophysical Supercomputing with GPUs: Critical Decisions for Early Adopters
• New Young Planetary Nebulae in IPHAS
• The Outer Halo of the Milky Way as Probed by RR Lyr Variables from the Palomar Transient Facility
• Automatic vetting of planet candidates from ground based surveys: Machine learning with NGTS
• Data-driven, interpretable photometric redshifts trained on heterogeneous and unrepresentative data
• Unsupervised Classification of Variable Stars
• Analysing Astronomy Algorithms for GPUs and Beyond
• A Machine Learning Artificial Neural Network Calibration of the Strong-Line Oxygen Abundance
• An artificial neural network to discover Hypervelocity stars: Candidates in Gaia DR1/TGAS
• Discovery of New Dipper Stars with K2: A Window into the Inner Disk Region of T Tauri Stars
• An autoencoder of stellar spectra and its application in automatically estimating atmospheric parameters
• An Updated Catalog of 4680 Northern Eclipsing Binaries with Algol-Type light curve morphology in the Catalina Sky Surveys
• Photometric Redshifts based on standard SED fitting procedures
• A novel single-pulse search approach to detection of dispersed radio pulses using clustering and supervised machine learning
• Estimating the Mass of the Local Group using Machine Learning Applied to Numerical Simulations
• Calibrating the HISA temperature: Measuring the temperature of the Riegel-Crutcher cloud
• Characterization of Near-Earth Asteroids using KMTNet-SAAO
• Statistical analysis of probability density functions for photometric redshifts through the KiDS-ESO-DR3 galaxies
• Effect of training characteristics on object classification: an application using Boosted Decision Trees
• Machine learning in APOGEE: Unsupervised spectral classification with $K$-means
• A Bayesian Framework for Cosmic String Searches in CMB Maps
• Predicting the Neutral Hydrogen Content of Galaxies From Optical Data Using Machine Learning
• Real-Time Data Mining of Massive Data Streams from Synoptic Sky Surveys
• Taxonomy and Light-Curve Data of 1000 Serendipitously Observed Main-Belt Asteroids
• Unsupervised feature-learning for galaxy SEDs with denoising autoencoders
• Inferring physical properties of galaxies from their emission line spectra
• Identifying Tidal Disruption Events via Prior Photometric Selection of Their Preferred Hosts
• Machine Learning Based Real Bogus System for HSC-SSP Moving Object Detecting Pipeline
• Automatic Survey-Invariant Variable Star Classification
• GAME: GAlaxy Machine learning for Emission lines
• Spectral feature extraction for DB white dwarfs through machine learning applied to new discoveries in the SDSS DR12 and DR14
• DAMEWARE: A web cyberinfrastructure for astrophysical data mining
• Characterizing the velocity of a wandering black hole and properties of the surrounding medium using convolutional neural networks
• Artificial Neural Network in Cosmic Landscape
• Sacrificing information for the greater good: how to select photometric bands for optimal accuracy
• Assessing the Performance of a Machine Learning Algorithm in Identifying Bubbles in Dust Emission
• Protostellar classification using supervised machine learning algorithms
• Using the Properties of Broad Absorption Line Quasars to Illuminate Quasar Structure
• Cosmic String Detection with Tree-Based Machine Learning
• Searching for Moving Objects in HSC-SSP: Pipeline and Preliminary Results
• Gaia eclipsing binary and multiple systems. Supervised classification and self-organizing maps
• Discovery, classification, and scientific exploration of transient events from the Catalina Real-time Transient Survey
• Information Retrieval and Recommendation System for Astronomical Observatories
• TSARDI: a Machine Learning data rejection algorithm for transiting exoplanet light curves
• Labeling Bias in Galaxy Morphologies
• Hubble Tarantula Treasury Project - VI. Identification of Pre-Main-Sequence Stars using Machine Learning techniques
• Applications of Machine-Learning Algorithms for Infrared Colour Selection of Galactic Wolf-Rayet Stars
• Improved Selection Criteria for HII Regions, based on IRAS Sources
• A Machine Learning Approach to Galaxy-LSS Classification I: Imprints on Halo Merger Trees
• Reliable mass calculation in spherical gravitating systems
• Detecting unresolved binary stars in Euclid VIS images
• Asteroids in the High cadence Transient Survey
• Galactic cartography with SkyMapper: I. Population sub-structure and the stellar number density of the inner halo
• SpArcFiRe: morphological selection effects due to reduced visibility of tightly winding arms in distant spiral galaxies
• Sharpening up Galactic all-sky maps with complementary data - A machine learning approach
• DAMEWARE - Data Mining & Exploration Web Application Resource

Cited by in corpus (49)

• Painting a portrait of the Galactic disc with its stellar clusters
• "Garbage In, Garbage Out" Revisited: What Do Machine Learning Application Papers Report About Human-Labeled Training Data?
• Full-sky Cosmic Microwave Background Foreground Cleaning Using Machine Learning
• Machine learning technique for morphological classification of galaxies from the SDSS. I. Photometry-based approach
• A machine learning approach to mapping baryons onto dark matter haloes using the EAGLE and C-EAGLE simulations
• On the discovery of stars, quasars, and galaxies in the Southern Hemisphere with S-PLUS DR2
• Transferring spectroscopic stellar labels to 217 million Gaia DR3 XP stars with SHBoost
• Applications of Machine Learning to Predicting Core-collapse Supernova Explosion Outcomes
• XLB: A differentiable massively parallel lattice Boltzmann library in Python
• Photometric redshifts with machine learning, lights and shadows on a complex data science use case
• Machine Learning applied to asteroid dynamics
• Classification of Planetary Nebulae through Deep Transfer Learning
• Searching for AGN and Pulsar Candidates in 4FGL Unassociated Sources Using Machine Learning
• Multi-Epoch Machine Learning 1: Unravelling Nature vs Nurture for Galaxy Formation
• PhotoRedshift-MML: a multimodal machine learning method for estimating photometric redshifts of quasars
• Hybrid analytic and machine-learned baryonic property insertion into galactic dark matter haloes
• Interpreting automatic AGN classifiers with saliency maps
• Quantifying the structure of strong gravitational lens potentials with uncertainty-aware deep neural networks
• Recovering Cosmic Microwave Background Polarization Signals with Machine Learning
• Analyzing the Galactic pulsar distribution with machine learning
• Mapping the X-ray variability of GRS1915+105 with machine learning
• Neural network-based emulation of interstellar medium models
• ULISSE: A Tool for One-shot Sky Exploration and its Application to Active Galactic Nuclei Detection
• A Comparison of Deep Learning Architectures for Optical Galaxy Morphology Classification
• A Census of Protostellar Outflows in Nearby Molecular Clouds
• Machine learning technique for morphological classification of galaxies from SDSS. II. The image-based morphological catalogs of galaxies at 0.02<z<0.1
• Uncovering Tidal Treasures: Automated Classification of Faint Tidal Features in DECaLS Data
• Measuring Young Stars in Space and Time -- II. The Pre-Main-Sequence Stellar Content of N44
• Identifying Potential Exomoon Signals with Convolutional Neural Networks
• Deep learning inference with the Event Horizon Telescope II. The Zingularity framework for Bayesian artificial neural networks
• CzSL: Learning from citizen science, experts and unlabelled data in astronomical image classification
• The Classification of Optical Galaxy Morphology Using Unsupervised Learning Techniques
• How to set up your first machine learning project in astronomy
• Lensing reconstruction from the cosmic microwave background polarization with machine learning
• Galaxy pairs in The Three Hundred simulations II: studying bound ones and identifying them via machine learning
• Rejection criteria based on outliers in the KiDS photometric redshifts and PDF distributions derived by machine learning
• A morphological segmentation approach to determining bar lengths
• Bayesian decomposition of the Galactic multi-frequency sky using probabilistic autoencoders
• StarUnLink: identifying and mitigating signals from communications satellites in stellar spectral surveys
• Participatory Science and Machine Learning Applied to Millions of Sources in the Hobby-Eberly Telescope Dark Energy Experiment
• Uncertainty Quantification of the Virial Black Hole Mass with Conformal Prediction
• Machine learning for exoplanet detection in high-contrast spectroscopy Combining cross correlation maps and deep learning on medium-resolution integral-field spectra
• Galactic Component Mapping of Galaxy UGC 2885 by Machine Learning Classification
• ULISSE: Determination of star-formation rate and stellar mass based on the one-shot galaxy imaging technique
• Gamma-ray burst light curve reconstruction with predictive models
• Prediction of Individual Halo Concentrations Across Cosmic Time Using Neural Networks
• A machine learning approach to estimate mid-infrared fluxes from WISE data
• A gradient boosting and broadband approach to finding Lyman-α emitting galaxies beyond narrowband surveys
• Bar Properties and Star-by-Star Bar Membership via Action Conservation