Astronomia ex machina: a history, primer, and outlook on neural networks in astronomy

Abstract

In this review, we explore the historical development and future prospects of artificial intelligence (AI) and deep learning in astronomy. We trace the evolution of connectionism in astronomy through its three waves, from the early use of multilayer perceptrons, to the rise of convolutional and recurrent neural networks, and finally to the current era of unsupervised and generative deep learning methods. With the exponential growth of astronomical data, deep learning techniques offer an unprecedented opportunity to uncover valuable insights and tackle previously intractable problems. As we enter the anticipated fourth wave of astronomical connectionism, we argue for the adoption of GPT-like foundation models fine-tuned for astronomical applications. Such models could harness the wealth of high-quality, multimodal astronomical data to serve state-of-the-art downstream tasks. To keep pace with advancements driven by Big Tech, we propose a collaborative, open-source approach within the astronomy community to develop and maintain these foundation models, fostering a symbiotic relationship between AI and astronomy that capitalizes on the unique strengths of both fields.

75 pages, 327 references, 32 figures. Review accepted in Royal Society Open Science

References in corpus (30)

• The Transiting Exoplanet Survey Satellite
• Overview of the SDSS-IV MaNGA Survey: Mapping Nearby Galaxies at Apache Point Observatory
• Rotation-invariant convolutional neural networks for galaxy morphology prediction
• LaMDA: Language Models for Dialog Applications
• Compute Trends Across Three Eras of Machine Learning
• Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
• Scaling Language Models: Methods, Analysis & Insights from Training Gopher
• Large-scale dark matter simulations
• Results from the Supernova Photometric Classification Challenge
• Predicting Solar Flares Using a Long Short-Term Memory Network
• Classifying Radio Galaxies with Convolutional Neural Network
• A recurrent neural network for classification of unevenly sampled variable stars
• The DAWES review 10: The impact of deep learning for the analysis of galaxy surveys
• Fast cosmological parameter estimation using neural networks
• {\sc CosmoNet}: fast cosmological parameter estimation in non-flat models using neural networks
• A sample of Seyfert-2 galaxies with ultra-luminous galaxy-wide NLRs -- Quasar light echos?
• Realistic galaxy image simulation via score-based generative models
• Denoising Gravitational Waves with Enhanced Deep Recurrent Denoising Auto-Encoders
• ASTROMER: A transformer-based embedding for the representation of light curves
• Probabilistic Mass Mapping with Neural Score Estimation
• Euclid preparation: XIII. Forecasts for galaxy morphology with the Euclid Survey using Deep Generative Models
• Detecting gravitational lenses using machine learning: exploring interpretability and sensitivity to rare lensing configurations
• Deep Neural Network Classifier for Variable Stars with Novelty Detection Capability
• Capturing the physics of MaNGA galaxies with self-supervised Machine Learning
• Use of neural networks for the identification of new z>=3.6 QSOs from FIRST-SDSS DR5
• Realistic galaxy images and improved robustness in machine learning tasks from generative modelling
• Super-resolving Herschel imaging: a proof of concept using Deep Neural Networks
• Foreground removal from CMB temperature maps using an MLP neural network
• Survey2Survey: A deep learning generative model approach for cross-survey image mapping
• Radio Galaxy Zoo EMU: Towards a Semantic Radio Galaxy Morphology Taxonomy

Cited by in corpus (20)

• Hydrodynamical simulations of the galaxy population: enduring successes and outstanding challenges
• A review of unsupervised learning in astronomy
• Towards an astronomical foundation model for stars with a Transformer-based model
• Empirical assessment of ChatGPT's answering capabilities in natural science and engineering
• Are Large Language Models a Threat to Digital Public Goods? Evidence from Activity on Stack Overflow
• Applications of machine learning in gravitational wave research with current interferometric detectors
• Deriving the star formation histories of galaxies from spectra with simulation-based inference
• Learning the Universe: Cosmological and Astrophysical Parameter Inference with Galaxy Luminosity Functions and Colours
• Inferring redshift and galaxy properties via a multi-task neural net with probabilistic outputs: An application to simulated MOONS spectra
• Deep learning inference with the Event Horizon Telescope II. The Zingularity framework for Bayesian artificial neural networks
• The Three Hundred Project: Mapping The Matter Distribution in Galaxy Clusters Via Deep Learning from Multiview Simulated Observations
• Morpho-Photometric Classification of KiDS DR5 Sources Based on Neural Networks: A Comprehensive Star-Quasar-Galaxy Catalog
• Simulating images of radio galaxies with diffusion models
• Uncertainty estimation for time series classification: Exploring predictive uncertainty in transformer-based models for variable stars
• Neutron star envelopes with machine learning: a single-hidden-layer neural network application
• Exploring Galaxy Properties of eCALIFA with Contrastive Learning
• Datacube segmentation via Deep Spectral Clustering
• The Galaxy Activity, Torus, and Outflow Survey (GATOS): N. Unveiling physical processes in local active galaxies. Unsupervised hierarchical clustering of JWST MIRI/MRS observations
• Deep Learning generated observations of galaxy clusters from dark-matter-only simulations
• EMU/GAMA: A new approach to characterising radio luminosity functions