A Comparison of Flare Forecasting Methods. IV. Evaluating Consecutive-Day Forecasting Patterns

Abstract

A crucial challenge to successful flare prediction is forecasting periods that transition between "flare-quiet" and "flare-active". Building on earlier studies in this series (Barnes et al. 2016; Leka et al. 2019a,b) in which we describe methodology, details, and results of flare forecasting comparison efforts, we focus here on patterns of forecast outcomes (success and failure) over multi-day periods. A novel analysis is developed to evaluate forecasting success in the context of catching the first event of flare-active periods, and conversely, of correctly predicting declining flare activity. We demonstrate these evaluation methods graphically and quantitatively as they provide both quick comparative evaluations and options for detailed analysis. For the testing interval 2016-2017, we determine the relative frequency distribution of two-day dichotomous forecast outcomes for three different event histories (i.e., event/event, no-event/event and event/no-event), and use it to highlight performance differences between forecasting methods. A trend is identified across all forecasting methods that a high/low forecast probability on day-1 remains high/low on day-2 even though flaring activity is transitioning. For M-class and larger flares, we find that explicitly including persistence or prior flare history in computing forecasts helps to improve overall forecast performance. It is also found that using magnetic/modern data leads to improvement in catching the first-event/first-no-event transitions. Finally, 15% of major (i.e., M-class or above) flare days over the testing interval were effectively missed due to a lack of observations from instruments away from the Earth-Sun line.

33 pages, 13 figures, accepted for publication in ApJ

References in corpus (10)

• Toward Reliable Benchmarking of Solar Flare Forecasting Methods
• A Comparison of Flare Forecasting Methods, I: Results from the "All-Clear" Workshop
• A Comparison of Flare Forecasting Methods. II. Benchmarks, Metrics and Performance Results for Operational Solar Flare Forecasting Systems
• Flare forecasting at the Met Office Space Weather Operations Centre
• Productivity of Solar Flares and Magnetic Helicity Injection in Active Regions
• A Comparison of Flare Forecasting Methods. III. Systematic Behaviors of Operational Solar Flare Forecasting Systems
• Flaring Rates and the Evolution of Sunspot Group McIntosh Classifications
• Verification of operational solar flare forecast: Case of Regional Warning Center Japan
• Verification of Space Weather Forecasts issued by the Met Office Space Weather Operations Centre
• Performance of Major Flare Watches from the Max Millennium Program (2001-2010)

Cited by in corpus (12)

• How to Train Your Flare Prediction Model: Revisiting Robust Sampling of Rare Events
• Operational solar flare prediction model using Deep Flare Net
• Implementation paradigm for supervised flare forecasting studies: a deep learning application with video data
• Reliable Probability Forecast of Solar Flares: Deep Flare Net-Reliable (DeFN-R)
• Decreasing False Alarm Rates in ML-based Solar Flare Prediction using SDO/HMI Data
• Flare Forecasting Algorithms Based on High-Gradient Polarity Inversion Lines in Active Regions
• A new catalogue of solar flare events from soft x-ray GOES signal in the period 1986-2020
• Self-consistent propagation of flux ropes in realistic coronal simulations
• The characteristics of flare- and CME-productive solar active regions
• Machine learning as a flaring storm warning machine: Was a warning machine for the September 2017 solar flaring storm possible?
• Comparison of magnetic energy and helicity in coronal jet simulations
• Comparative case study of two methods to assess the eruptive potential of selected active regions