Long-term series forecasting with Query Selector -- efficient model of sparse attention

Abstract

Various modifications of TRANSFORMER were recently used to solve time-series forecasting problem. We propose Query Selector - an efficient, deterministic algorithm for sparse attention matrix. Experiments show it achieves state-of-the art results on ETT, Helpdesk and BPI'12 datasets.

10 pages

References in corpus (4)

• Generating Long Sequences with Sparse Transformers
• Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks
• ZeRO-Offload: Democratizing Billion-Scale Model Training
• Sequence to Sequence with Attention for Influenza Prevalence Prediction using Google Trends