Deep Learning for Time Series Forecasting: Tutorial and Literature Survey

Abstract

Deep learning based forecasting methods have become the methods of choice in many applications of time series prediction or forecasting often outperforming other approaches. Consequently, over the last years, these methods are now ubiquitous in large-scale industrial forecasting applications and have consistently ranked among the best entries in forecasting competitions (e.g., M4 and M5). This practical success has further increased the academic interest to understand and improve deep forecasting methods. In this article we provide an introduction and overview of the field: We present important building blocks for deep forecasting in some depth; using these building blocks, we then survey the breadth of the recent deep forecasting literature.

33 pages, 6 figures

References in corpus (51)

• XGBoost: A Scalable Tree Boosting System
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Neural Machine Translation by Jointly Learning to Align and Translate
• TensorFlow: A system for large-scale machine learning
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• On the difficulty of training Recurrent Neural Networks
• WaveNet: A Generative Model for Raw Audio
• Self-Attention Generative Adversarial Networks
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Time Series Forecasting With Deep Learning: A Survey
• Attention-Based Models for Speech Recognition
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics
• Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions
• Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• C-RNN-GAN: Continuous recurrent neural networks with adversarial training
• Conditional Time Series Forecasting with Convolutional Neural Networks
• Deep and Confident Prediction for Time Series at Uber
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Transfer learning for time series classification
• A Multi-Horizon Quantile Recurrent Forecaster
• Neural Relational Inference for Interacting Systems
• GANSynth: Adversarial Neural Audio Synthesis
• Unsupervised Scalable Representation Learning for Multivariate Time Series
• LSTM-MSNet: Leveraging Forecasts on Sets of Related Time Series with Multiple Seasonal Patterns
• The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process
• A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
• Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting
• How to Train Your Energy-Based Models
• Multivariate Probabilistic Time Series Forecasting via Conditioned Normalizing Flows
• Wasserstein Learning of Deep Generative Point Process Models
• Deep Factors for Forecasting
• Forecasting: theory and practice
• Joint Modeling of Event Sequence and Time Series with Attentional Twin Recurrent Neural Networks
• Neural Flows: Efficient Alternative to Neural ODEs
• Foundations of Sequence-to-Sequence Modeling for Time Series
• Effective Bayesian Modeling of Groups of Related Count Time Series
• Bridging Physics-based and Data-driven modeling for Learning Dynamical Systems
• Parsimonious Quantile Regression of Financial Asset Tail Dynamics via Sequential Learning
• PSA-GAN: Progressive Self Attention GANs for Synthetic Time Series
• Deep Generative Quantile-Copula Models for Probabilistic Forecasting
• Multivariate Time Series Forecasting with Latent Graph Inference
• Learning Quantile Functions without Quantile Crossing for Distribution-free Time Series Forecasting
• Probabilistic Time Series Forecasting with Implicit Quantile Networks
• Multivariate Quantile Function Forecaster
• Multi-Objective Model Selection for Time Series Forecasting
• A Shooting Formulation of Deep Learning
• Deep Explicit Duration Switching Models for Time Series
• A Study of Joint Graph Inference and Forecasting
• A simple and effective predictive resource scaling heuristic for large-scale cloud applications
• Detecting Anomalous Event Sequences with Temporal Point Processes

Cited by in corpus (16)

• Deep Learning for Time Series Forecasting: A Survey
• Short-Term Electricity-Load Forecasting by Deep Learning: A Comprehensive Survey
• How Does It Function? Characterizing Long-term Trends in Production Serverless Workloads
• The Digital Transformation in Health: How AI Can Improve the Performance of Health Systems
• Graph Deep Learning for Time Series Forecasting
• Time-Series Forecasting in Smart Manufacturing Systems: An Experimental Evaluation of the State-of-the-art Algorithms
• Wind energy forecasting with missing values within a fully conditional specification framework
• Managing Cold-start in The Serverless Cloud with Temporal Convolutional Networks
• Midwifery Learning and Forecasting: Predicting Content Demand with User-Generated Logs
• Multiscale Spatio-Temporal Enhanced Short-term Load Forecasting of Electric Vehicle Charging Stations
• Comparative analysis and evaluation of ageing forecasting methods for semiconductor devices in online health monitoring
• A Survey on Event Prediction Methods from a Systems Perspective: Bringing Together Disparate Research Areas
• A Model-Constrained Tangent Slope Learning Approach for Dynamical Systems
• Physical Self-Supervised Learning: IMU Sensing without Manual Labels
• Integrating Time Series into LLMs via Multi-layer Steerable Embedding Fusion for Enhanced Forecasting
• Interpretable deep convolutional model for nonlinear multivariate time series in complex systems