Decreasing False Alarm Rates in ML-based Solar Flare Prediction using SDO/HMI Data

Abstract

A hybrid two-stage machine learning architecture that addresses the problem of excessive false positives (false alarms) in solar flare prediction systems is investigated. The first stage is a convolutional neural network (CNN) model based on the VGG-16 architecture that extracts features from a temporal stack of consecutive Solar Dynamics Observatory (SDO) Helioseismic and Magnetic Imager (HMI) magnetogram images to produce a flaring probability. The probability of flaring is added to a feature vector derived from the magnetograms to train an extremely randomized trees (ERT) model in the second stage to produce a binary deterministic prediction (flare/no flare) in a 12-hour forecast window. To tune the hyperparameters of the architecture a new evaluation metric is introduced, the "scaled True Skill Statistic". It specifically addresses the large discrepancy between the true positive rate and the false positive rate in the highly unbalanced solar flare event training datasets. Through hyperparameter tuning to maximize this new metric, our two-stage architecture drastically reduces false positives by $\approx$ $48\%$ without significantly affecting the true positives (reduction by $\approx$ $12\%$), when compared with predictions from the first stage CNN alone. This, in turn, improves various traditional binary classification metrics sensitive to false positives such as the precision, F1 and the Heidke Skill Score. The end result is a more robust 12-hour flare prediction system that could be combined with current operational flare forecasting methods. Additionally, using the ERT-based feature ranking mechanism, we show that the CNN output probability is highly ranked in terms of flare prediction relevance.

References in corpus (8)

• The Effectiveness of Data Augmentation in Image Classification using Deep Learning
• The Helioseismic and Magnetic Imager (HMI) Vector Magnetic Field Pipeline: SHARPs -- Space-weather HMI Active Region Patches
• Solar Flare Prediction Using SDO/HMI Vector Magnetic Field Data with a Machine-Learning Algorithm
• A Comparison of Flare Forecasting Methods, I: Results from the "All-Clear" Workshop
• A Comparison of Flare Forecasting Methods. IV. Evaluating Consecutive-Day Forecasting Patterns
• SMARPs and SHARPs: Two Solar Cycles of Active Region Data
• Leveraging the Mathematics of Shape for Solar Magnetic Eruption Prediction
• Direct observation of the energy release site in a solar flare by SDO/AIA, Hinode/EIS and RHESSI

Cited by in corpus (4)

• Identifying preflare spectral features using explainable artificial intelligence
• Machine Learning Approaches to Solar-Flare Forecasting: Is Complex Better?
• Solar Active Region Magnetogram Image Dataset for Studies of Space Weather
• Verification of the NOAA Space Weather Prediction Center solar flare forecast (1998-2024)