The Challenge of Machine Learning in Space Weather Nowcasting and Forecasting

Abstract

The numerous recent breakthroughs in machine learning (ML) make imperative to carefully ponder how the scientific community can benefit from a technology that, although not necessarily new, is today living its golden age. This Grand Challenge review paper is focused on the present and future role of machine learning in space weather. The purpose is twofold. On one hand, we will discuss previous works that use ML for space weather forecasting, focusing in particular on the few areas that have seen most activity: the forecasting of geomagnetic indices, of relativistic electrons at geosynchronous orbits, of solar flares occurrence, of coronal mass ejection propagation time, and of solar wind speed. On the other hand, this paper serves as a gentle introduction to the field of machine learning tailored to the space weather community and as a pointer to a number of open challenges that we believe the community should undertake in the next decade. The recurring themes throughout the review are the need to shift our forecasting paradigm to a probabilistic approach focused on the reliable assessment of uncertainties, and the combination of physics-based and machine learning approaches, known as gray-box.

under review

References in corpus (10)

• Explaining and Harnessing Adversarial Examples
• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• Solar Flare Prediction Using SDO/HMI Vector Magnetic Field Data with a Machine-Learning Algorithm
• Toward Reliable Benchmarking of Solar Flare Forecasting Methods
• Introduction to astroML: Machine Learning for Astrophysics
• Solar Flare Prediction Model with Three Machine-Learning Algorithms Using Ultraviolet Brightening and Vector Magnetogram
• Obtaining Calibrated Probabilities from Boosting
• Searching for Exoplanets Using Artificial Intelligence
• Classification of Solar Wind with Machine Learning
• Snakes on a Spaceship - An Overview of Python in Heliophysics

Cited by in corpus (50)

• What Sustained Multi-Disciplinary Research Can Achieve: The Space Weather Modeling Framework
• Identifying Solar Flare Precursors Using Time Series of SDO/HMI Images and SHARP Parameters
• The data-driven future of high energy density physics
• Relativistic electron flux model in the outer radiation belt using a neural network approach
• How to Train Your Flare Prediction Model: Revisiting Robust Sampling of Rare Events
• Solar Flare Intensity Prediction with Machine Learning Models
• Forecasting Megaelectron-Volt Electrons inside Earth's Outer Radiation Belt: PreMevE 2.0 Based on Supervised Machine Learning Algorithms
• Deep Learning Enhanced Dynamic Mode Decomposition
• Dynamic Time Warping as a New Evaluation for Dst Forecast with Machine Learning
• Machine Learning in Orbit Estimation: a Survey
• Multi-Hour Ahead Dst Index Prediction Using Multi-Fidelity Boosted Neural Networks
• Collection, Collation, and Comparison of 3D Coronal CME Reconstructions
• Symplectic Learning for Hamiltonian Neural Networks
• Toward a Next Generation Particle Precipitation Model: Mesoscale Prediction Through Machine Learning (a Case Study and Framework for Progress)
• A gray-box model for a probabilistic estimate of regional ground magnetic perturbations: Enhancing the NOAA operational Geospace model with machine learning
• Opening the Black Box of the Radiation Belt Machine Learning Model
• Incorporating Physical Knowledge into Machine Learning for Planetary Space Physics
• Using gradient boosting regression to improve ambient solar wind model predictions
• Predicting Solar Wind Streams from the Inner-Heliosphere to Earth via Shifted Operator Inference
• Predicting Swarm Equatorial Plasma Bubbles via Machine Learning and Shapley Values
• Novel data analysis techniques in coronal seismology
• Low Dimensional Convolutional Neural Network For Solar Flares GOES Time Series Classification
• Leveraging the Mathematics of Shape for Solar Magnetic Eruption Prediction
• Forecasting GICs and geoelectric fields from solar wind data using LSTMs: application in Austria
• LeHaMoC: a versatile time-dependent lepto-hadronic modeling code for high-energy astrophysical sources
• Parameter Distributions for the Drag-Based Modeling of CME Propagation
• Identifying Flux Rope Signatures Using a Deep Neural Network
• Probabilistic prediction of Dst storms one-day-ahead using Full-Disk SoHO Images
• Timing of the solar wind propagation delay between L1 and Earth based on machine learning
• Unsupervised classification of simulated magnetospheric regions
• Prediction and understanding of soft proton contamination in XMM-Newton: a machine learning approach
• Global geomagnetic perturbation forecasting using Deep Learning
• Improving the Arrival Time Estimates of Coronal Mass Ejections by Using Magnetohydrodynamic Ensemble Modeling, Heliospheric Imager data, and Machine Learning
• Bayesian inference of quasi-linear radial diffusion parameters using Van Allen Probes
• Identification of high order closure terms from fully kinetic simulations using machine learning
• Analysis of SEP events and their possible precursors based on the GSEP Catalog
• Verification of the NOAA Space Weather Prediction Center solar flare forecast (1998-2024)
• Satellite galaxies' drag on field stars in the Milky Way
• Space weather: the solar perspective -- an update to Schwenn (2006)
• Exploring the potential of neural networks to predict statistics of solar wind turbulence
• A catalogue of observed geo-effective CME/ICME characteristics
• Extended Drag-Based Model for better predicting the evolution of Coronal Mass Ejections
• A Machine-Learning-Ready Dataset Prepared from the Solar and Heliospheric Observatory Mission
• Interpreting LSTM Prediction on Solar Flare Eruption with Time-series Clustering
• Forecasting Thermospheric Density with Transformers for Multi-Satellite Orbit Management
• Probabilistic Prediction of Geomagnetic Storms and the K${}_{\text{p}}$ Index
• The Flare Likelihood and Region Eruption Forecasting (FLARECAST) Project: Flare forecasting in the big data & machine learning era
• Earth-affecting Solar Transients: A Review of Progresses in Solar Cycle 24
• Beyond Mean Solar Wind Conditions: Turbulence-Aware Forecasting of the AE Index
• Testing and Validating Two Morphological Flare Predictors by Logistic Regression Machine Learning