Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions

Abstract

Recurrent Neural Networks (RNN) have become competitive forecasting methods, as most notably shown in the winning method of the recent M4 competition. However, established statistical models such as ETS and ARIMA gain their popularity not only from their high accuracy, but they are also suitable for non-expert users as they are robust, efficient, and automatic. In these areas, RNNs have still a long way to go. We present an extensive empirical study and an open-source software framework of existing RNN architectures for forecasting, that allow us to develop guidelines and best practices for their use. For example, we conclude that RNNs are capable of modelling seasonality directly if the series in the dataset possess homogeneous seasonal patterns, otherwise we recommend a deseasonalization step. Comparisons against ETS and ARIMA demonstrate that the implemented (semi-)automatic RNN models are no silver bullets, but they are competitive alternatives in many situations.

References in corpus (12)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Sequence to Sequence Learning with Neural Networks
• Practical Bayesian Optimization of Machine Learning Algorithms
• WaveNet: A Generative Model for Raw Audio
• Deep and Confident Prediction for Time Series at Uber
• TensorFlow Distributions
• Learning Stochastic Recurrent Networks
• Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting
• Capacity and Trainability in Recurrent Neural Networks
• GluonTS: Probabilistic Time Series Models in Python
• Deep Factors for Forecasting
• Towards a Deep Reinforcement Learning Approach for Tower Line Wars

Cited by in corpus (57)

• Deep Learning for Time Series Forecasting: Tutorial and Literature Survey
• LSTM-MSNet: Leveraging Forecasts on Sets of Related Time Series with Multiple Seasonal Patterns
• Foundation Models for Time Series Analysis: A Tutorial and Survey
• Deep learning for temporal data representation in electronic health records: A systematic review of challenges and methodologies
• A review of predictive uncertainty estimation with machine learning
• Tire Force Estimation in Intelligent Tires Using Machine Learning
• Evaluation of post-hoc interpretability methods in time-series classification
• Time Series Quantum Reservoir Computing with Weak and Projective Measurements
• It's All in the Embedding! Fake News Detection Using Document Embeddings
• Federated Learning based Energy Demand Prediction with Clustered Aggregation
• Ensembles of Localised Models for Time Series Forecasting
• Weather-Informed Probabilistic Forecasting and Scenario Generation in Power Systems
• Knowledge-based Deep Learning for Modeling Chaotic Systems
• W-Transformers : A Wavelet-based Transformer Framework for Univariate Time Series Forecasting
• Self-Supervised Learning for Data Scarcity in a Fatigue Damage Prognostic Problem
• Time series forecasting with Gaussian Processes needs priors
• Forecasting: theory and practice
• Time-Series Forecasting in Smart Manufacturing Systems: An Experimental Evaluation of the State-of-the-art Algorithms
• LAVARNET: Neural Network Modeling of Causal Variable Relationships for Multivariate Time Series Forecasting
• An Accurate and Fully-Automated Ensemble Model for Weekly Time Series Forecasting
• PyLightcurve-torch: a transit modelling package for deep learning applications in PyTorch
• Simulation and Optimisation of Air Conditioning Systems using Machine Learning
• Error-feedback stochastic modeling strategy for time series forecasting with convolutional neural networks
• Review of Time Series Forecasting Methods and Their Applications to Particle Accelerators
• Differentially Private Multivariate Time Series Forecasting of Aggregated Human Mobility With Deep Learning: Input or Gradient Perturbation?
• Gradient-Free Training of Recurrent Neural Networks using Random Perturbations
• Non-contrastive representation learning for intervals from well logs
• Predicting Stock Price Movement as an Image Classification Problem
• Learning Differential Operators for Interpretable Time Series Modeling
• Neural Chronos ODE: Unveiling Temporal Patterns and Forecasting Future and Past Trends in Time Series Data
• Convolutional Neural Networks for Searching Superflares from Pixel-level Data of the Transiting Exoplanet Survey Satellite
• The Importance of Environmental Factors in Forecasting Australian Power Demand
• An experimental evaluation of choices of SSA forecasting parameters
• Orbit: Probabilistic Forecast with Exponential Smoothing
• Forecasting with sktime: Designing sktime's New Forecasting API and Applying It to Replicate and Extend the M4 Study
• The Effectiveness of Discretization in Forecasting: An Empirical Study on Neural Time Series Models
• MixSeq: Connecting Macroscopic Time Series Forecasting with Microscopic Time Series Data
• Combining Forecasts using Meta-Learning: A Comparative Study for Complex Seasonality
• Large-scale End-of-Life Prediction of Hard Disks in Distributed Datacenters
• A review of machine learning concepts and methods for addressing challenges in probabilistic hydrological post-processing and forecasting
• Wasserstein GAN: Deep Generation applied on Bitcoins financial time series
• Towards Accurate Predictions and Causal 'What-if' Analyses for Planning and Policy-making: A Case Study in Emergency Medical Services Demand
• ML-Based Bidding Price Prediction for Pay-As-Bid Ancillary Services Markets: A Use Case in the German Control Reserve Market
• MMformer with Adaptive Transferable Attention: Advancing Multivariate Time Series Forecasting for Environmental Applications
• For2For: Learning to forecast from forecasts
• Hierarchical spline for time series forecasting: An application to Naval ship engine failure rate
• Social-Media Activity Forecasting with Exogenous Information Signals
• A Formal Language Approach to Explaining RNNs
• A Hybrid Residual Dilated LSTM end Exponential Smoothing Model for Mid-Term Electric Load Forecasting
• Two ways towards combining Sequential Neural Network and Statistical Methods to Improve the Prediction of Time Series
• Ensembles of Randomized NNs for Pattern-based Time Series Forecasting
• RNN-BOF: A Multivariate Global Recurrent Neural Network for Binary Outcome Forecasting of Inpatient Aggression
• Designing a double deep reinforcement learning selection tool for resilient demand prediction
• POLA: Online Time Series Prediction by Adaptive Learning Rates
• Optimizing LSTM Neural Networks for Resource-Constrained Retail Sales Forecasting: A Model Compression Study
• Temporally Folded Convolutional Neural Networks for Sequence Forecasting
• A Multi-criteria Approach to Evolve Sparse Neural Architectures for Stock Market Forecasting