Long-term series forecasting with Query Selector -- efficient model of sparse attention
arXiv:2107.08687
Abstract
Various modifications of TRANSFORMER were recently used to solve time-series forecasting problem. We propose Query Selector - an efficient, deterministic algorithm for sparse attention matrix. Experiments show it achieves state-of-the art results on ETT, Helpdesk and BPI'12 datasets.
arXiv comment10 pages
References in corpus (10)
- Longformer: The Long-Document Transformer
- Predictive Business Process Monitoring with LSTM Neural Networks
- Generating Long Sequences with Sparse Transformers
- A Multi-Horizon Quantile Recurrent Forecaster
- Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks
- Long-term Forecasting using Higher Order Tensor RNNs
- ZeRO-Offload: Democratizing Billion-Scale Model Training
- Do We Really Need Deep Learning Models for Time Series Forecasting?
- Time and Activity Sequence Prediction of Business Process Instances
- Sequence to Sequence with Attention for Influenza Prevalence Prediction using Google Trends