Proper Scoring Rules for Multivariate Probabilistic Forecasts based on Aggregation and Transformation

Abstract

Proper scoring rules are an essential tool to assess the predictive performance of probabilistic forecasts. However, propriety alone does not ensure an informative characterization of predictive performance and it is recommended to compare forecasts using multiple scoring rules. With that in mind, interpretable scoring rules providing complementary information are necessary. We formalize a framework based on aggregation and transformation to build interpretable multivariate proper scoring rules. Aggregation-and-transformation-based scoring rules are able to target specific features of the probabilistic forecasts; which improves the characterization of the predictive performance. This framework is illustrated through examples taken from the literature and studied using numerical experiments showcasing its benefits. In particular, it is shown that it can help bridge the gap between proper scoring rules and spatial verification tools.

arXiv commentfor associated code, see https://github.com/pic-romain/aggregation-transformation

References in corpus (18)

• Neural networks for post-processing ensemble weather forecasts
• FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators
• Wind Energy: Forecasting Challenges for Its Operational Management
• Uncertainty Quantification in Complex Simulation Models Using Ensemble Copula Coupling
• Statistical Postprocessing for Weather Forecasts -- Review, Challenges and Avenues in a Big Data World
• The rise of data-driven weather forecasting
• Forecasting Global Weather with Graph Neural Networks
• Moments and Absolute Moments of the Normal Distribution
• Comparison of nonhomogeneous regression models for probabilistic wind speed forecasting
• Reliability, Sufficiency, and the Decomposition of Proper Scores
• Theory and Applications of Proper Scoring Rules
• Probabilistic wind speed forecasting on a grid based on ensemble model output statistics
• Proper local scoring rules
• Minimum scoring rule inference
• The role of the information set for forecasting - with applications to risk management
• Local proper scoring rules of order two
• Why scoring functions cannot assess tail properties
• Power variations for fractional type infinitely divisible random fields

Cited by in corpus (4)

• Generative Machine Learning for Multivariate Angular Simulation
• Multivariate Uncertainty Quantification with Tomographic Quantile Forests
• Influence of river incision on landslides triggered in Nepal by the Gorkha earthquake: Results from a pixel-based susceptibility model using inlabru
• Variable transformations in consistent loss functions