A Comparison of Flare Forecasting Methods. III. Systematic Behaviors of Operational Solar Flare Forecasting Systems

Abstract

A workshop was recently held at Nagoya University (31 October - 02 November 2017), sponsored by the Center for International Collaborative Research, at the Institute for Space-Earth Environmental Research, Nagoya University, Japan, to quantitatively compare the performance of today's operational solar flare forecasting facilities. Building upon Paper I of this series (Barnes et al. 2016), in Paper II (Leka et al. 2019) we described the participating methods for this latest comparison effort, the evaluation methodology, and presented quantitative comparisons. In this paper we focus on the behavior and performance of the methods when evaluated in the context of broad implementation differences. Acknowledging the short testing interval available and the small number of methods available, we do find that forecast performance: 1) appears to improve by including persistence or prior flare activity, region evolution, and a human "forecaster in the loop"; 2) is hurt by restricting data to disk-center observations; 3) may benefit from long-term statistics, but mostly when then combined with modern data sources and statistical approaches. These trends are arguably weak and must be viewed with numerous caveats, as discussed both here and in Paper II. Following this present work, we present in Paper IV a novel analysis method to evaluate temporal patterns of forecasting errors of both types (i.e., misses and false alarms; Park et al. 2019). Hence, most importantly, with this series of papers we demonstrate the techniques for facilitating comparisons in the interest of establishing performance-positive methodologies.

arXiv comment23 pages, 6 figures, accepted for publication in The Astrophysical Journal

References in corpus (13)

• The Helioseismic and Magnetic Imager (HMI) Vector Magnetic Field Pipeline: SHARPs -- Space-weather HMI Active Region Patches
• The Helioseismic and Magnetic Imager (HMI) Vector Magnetic Field Pipeline: Overview and Performance
• Toward Reliable Benchmarking of Solar Flare Forecasting Methods
• A Comparison of Flare Forecasting Methods, I: Results from the "All-Clear" Workshop
• A Comparison of Flare Forecasting Methods. II. Benchmarks, Metrics and Performance Results for Operational Solar Flare Forecasting Systems
• The Helioseismic and Magnetic Imager (HMI) Vector Magnetic Field Pipeline: Optimization of the Spectral Line Inversion Code
• Flare forecasting at the Met Office Space Weather Operations Centre
• Productivity of Solar Flares and Magnetic Helicity Injection in Active Regions
• Flaring Rates and the Evolution of Sunspot Group McIntosh Classifications
• Evaluating (and Improving) Estimates of the Solar Radial Magnetic Field Component from Line-of-Sight Magnetograms
• Verification of operational solar flare forecast: Case of Regional Warning Center Japan
• Verification of Space Weather Forecasts issued by the Met Office Space Weather Operations Centre
• Performance of Major Flare Watches from the Max Millennium Program (2001-2010)

Cited by in corpus (20)

• A Comparison of Flare Forecasting Methods. II. Benchmarks, Metrics and Performance Results for Operational Solar Flare Forecasting Systems
• How to Train Your Flare Prediction Model: Revisiting Robust Sampling of Rare Events
• Feature ranking of active region source properties in solar flare forecasting and the uncompromised stochasticity of flare occurrence
• Solar Flare Intensity Prediction with Machine Learning Models
• A Comparison of Flare Forecasting Methods. IV. Evaluating Consecutive-Day Forecasting Patterns
• Implementation paradigm for supervised flare forecasting studies: a deep learning application with video data
• Decreasing False Alarm Rates in ML-based Solar Flare Prediction using SDO/HMI Data
• Flare Forecasting Algorithms Based on High-Gradient Polarity Inversion Lines in Active Regions
• Signature of the turbulent component of solar dynamo on active region scales and its association with flaring activity
• Leveraging the Mathematics of Shape for Solar Magnetic Eruption Prediction
• Solar Flares Forecasting Using Time Series and Extreme Gradient Boosting Ensembles
• Properties of Flare-Imminent versus Flare-Quiet Active Regions from the Chromosphere through the Corona II: NonParametric Discriminant Analysis Results from the NWRA Classification Infrastructure (NCI)
• Machine Learning Approaches to Solar-Flare Forecasting: Is Complex Better?
• The characteristics of flare- and CME-productive solar active regions
• Comparison of magnetic energy and helicity in coronal jet simulations
• Planetary statistics and forecasting for solar flares
• Evaluating Pointing Strategies for Future Solar Flare Missions
• Comparative case study of two methods to assess the eruptive potential of selected active regions
• Testing and Validating Two Morphological Flare Predictors by Logistic Regression Machine Learning
• Forecasting Solar Flares by Data Assimilation in Sandpile Models