XGBoostLSS -- An extension of XGBoost to probabilistic forecasting

Abstract

We propose a new framework of XGBoost that predicts the entire conditional distribution of a univariate response variable. In particular, XGBoostLSS models all moments of a parametric distribution (i.e., mean, location, scale and shape [LSS]) instead of the conditional mean only. Choosing from a wide range of continuous, discrete and mixed discrete-continuous distribution, modelling and predicting the entire conditional distribution greatly enhances the flexibility of XGBoost, as it allows to gain additional insight into the data generating process, as well as to create probabilistic forecasts from which prediction intervals and quantiles of interest can be derived. We present both a simulation study and real world examples that demonstrate the virtues of our approach.

arXiv commentBayesian Optimization; Distributional Modeling; Expectile Regression; GAMLSS; Probabilistic Forecast; Uncertainty Quantification; XGBoost

References in corpus (8)

• Consistent Individualized Feature Attribution for Tree Ensembles
• Bayesian structured additive distributional regression with an application to regional income inequality in Germany
• mlrMBO: A Modular Framework for Model-Based Optimization of Expensive Black-Box Functions
• Stability selection for component-wise gradient boosting in multiple dimensions
• Distributional Regression Forests for Probabilistic Precipitation Forecasting in Complex Terrain
• Heteroscedastic BART Using Multiplicative Regression Trees
• Transformation Forests
• Multivariate Conditional Transformation Models

Cited by in corpus (3)

• M5 Competition Uncertainty: Overdispersion, distributional forecasting, GAMLSS and beyond
• CatBoostLSS -- An extension of CatBoost to probabilistic forecasting
• TreeFlow: Going beyond Tree-based Gaussian Probabilistic Regression