Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case

Abstract

In this paper, we present a new approach to time series forecasting. Time series data are prevalent in many scientific and engineering disciplines. Time series forecasting is a crucial task in modeling time series data, and is an important area of machine learning. In this work we developed a novel method that employs Transformer-based machine learning models to forecast time series data. This approach works by leveraging self-attention mechanisms to learn complex patterns and dynamics from time series data. Moreover, it is a generic framework and can be applied to univariate and multivariate time series data, as well as time series embeddings. Using influenza-like illness (ILI) forecasting as a case study, we show that the forecasting results produced by our approach are favorably comparable to the state-of-the-art.

arXiv comment10 pages, 7 figures

References in corpus (1)

• Accurate estimation of influenza epidemics using Google search data via ARGO

Cited by in corpus (29)

• Self-Supervised Representation Learning: Introduction, Advances and Challenges
• Dual Aspect Self-Attention based on Transformer for Remaining Useful Life Prediction
• TCCT: Tightly-Coupled Convolutional Transformer on Time Series Forecasting
• DTAAD: Dual Tcn-Attention Networks for Anomaly Detection in Multivariate Time Series Data
• Annotating Columns with Pre-trained Language Models
• Using deep learning to understand and mitigate the qubit noise environment
• Knowledge-based Deep Learning for Modeling Chaotic Systems
• W-Transformers : A Wavelet-based Transformer Framework for Univariate Time Series Forecasting
• Epicasting: An Ensemble Wavelet Neural Network (EWNet) for Forecasting Epidemics
• Time Series Analysis and Modeling to Forecast: a Survey
• A Neural Network Transformer Model for Composite Microstructure Homogenization
• Representation learning for neural population activity with Neural Data Transformers
• NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting
• Bridging Physics-based and Data-driven modeling for Learning Dynamical Systems
• MAD: Self-Supervised Masked Anomaly Detection Task for Multivariate Time Series
• Pose2Trajectory: Using Transformers on Body Pose to Predict Tennis Player's Trajectory
• TraDE: Transformers for Density Estimation
• Reverse Ordering Techniques for Attention-Based Channel Prediction
• Automated Essay Scoring Using Transformer Models
• MixSeq: Connecting Macroscopic Time Series Forecasting with Microscopic Time Series Data
• Transformer Networks for Data Augmentation of Human Physical Activity Recognition
• A Spatio-temporal Prediction Methodology Based on Deep Learning and Real Wi-Fi Measurements
• Limited Resource Allocation in a Non-Markovian World: The Case of Maternal and Child Healthcare
• Am I fit for this physical activity? Neural embedding of physical conditioning from inertial sensors
• On-device AI: Quantization-aware Training of Transformers in Time-Series
• Surrogate Modeling for Scalable Evaluation of Distributed Computing Systems for HEP Applications
• An early prediction of covid-19 associated hospitalization surge using deep learning approach
• Modelling stochastic time delay for regression analysis
• Dynamic Virtual Graph Significance Networks for Predicting Influenza