A Worrying Analysis of Probabilistic Time-series Models for Sales Forecasting
arXiv:2011.10715
Abstract
Probabilistic time-series models become popular in the forecasting field as they help to make optimal decisions under uncertainty. Despite the growing interest, a lack of thorough analysis hinders choosing what is worth applying for the desired task. In this paper, we analyze the performance of three prominent probabilistic time-series models for sales forecasting. To remove the role of random chance in architecture's performance, we make two experimental principles; 1) Large-scale dataset with various cross-validation sets. 2) A standardized training and hyperparameter selection. The experimental results show that a simple Multi-layer Perceptron and Linear Regression outperform the probabilistic models on RMSE without any feature engineering. Overall, the probabilistic models fail to achieve better performance on point estimation, such as RMSE and MAPE, than comparably simple baselines. We analyze and discuss the performances of probabilistic time-series models.
arXiv commentNeurIPS 2020 workshop (I Can't Believe It's Not Better, ICBINB@NeurIPS 2020). All authors contributed equally to this research
References in corpus (9)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Sequence to Sequence Learning with Neural Networks
- Deep Kalman Filters
- GluonTS: Probabilistic Time Series Models in Python
- Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data
- Deep Factors for Forecasting
- High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes
- Adaptive Path-Integral Autoencoder: Representation Learning and Planning for Dynamical Systems
- Probabilistic Recurrent State-Space Models