Point forecasting and forecast evaluation with generalized Huber loss

Abstract

Huber loss, its asymmetric variants and their associated functionals (here named Huber functionals) are studied in the context of point forecasting and forecast evaluation. The Huber functional of a distribution is the set of minimizers of the expected (asymmetric) Huber loss, is an intermediary between a quantile and corresponding expectile, and also arises in M-estimation. Each Huber functional is elicitable, generating the precise set of minimizers of an expected score, subject to weak regularity conditions on the class of probability distributions, and has a complete characterization of its consistent scoring functions. Such scoring functions admit a mixture representation as a weighted average of elementary scoring functions. Each elementary score can be interpreted as the relative economic loss of using a particular forecast for a class of investment decisions where profits and losses are capped. The relevance of this theory for comparative assessment of weather forecasts is also discussed.

25 pages, 4 figures, 2 tables. No changes to core mathematical results. Latest version includes more context on applications, and removes section on robust forecast verification of forecasts targeting the mean functional

References in corpus (4)

• Elicitability and its Application in Risk Management
• A scoring framework for tiered warnings and multicategorical forecasts based on fixed risk measures
• Evaluation of point forecasts for extreme events using consistent scoring functions
• Robust Estimation and Shrinkage in Ultrahigh Dimensional Expectile Regression with Heavy Tails and Variance Heterogeneity

Cited by in corpus (9)

• A review of predictive uncertainty estimation with machine learning
• Regression Diagnostics meets Forecast Evaluation: Conditional Calibration, Reliability Diagrams, and Coefficient of Determination
• Characterizing M-estimators
• scores: A Python package for verifying and evaluating models and predictions with xarray
• Scalable stellar evolution forecasting: Deep learning emulation vs. hierarchical nearest neighbor interpolation
• Deep Huber quantile regression networks
• A scoring framework for tiered warnings and multicategorical forecasts based on fixed risk measures
• Evaluation of point forecasts for extreme events using consistent scoring functions
• Variable transformations in consistent loss functions