NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting
arXiv:2102.05624
Abstract
Although Transformer has made breakthrough success in widespread domains especially in Natural Language Processing (NLP), applying it to time series forecasting is still a great challenge. In time series forecasting, the autoregressive decoding of canonical Transformer models could introduce huge accumulative errors inevitably. Besides, utilizing Transformer to deal with spatial-temporal dependencies in the problem still faces tough difficulties.~To tackle these limitations, this work is the first attempt to propose a Non-Autoregressive Transformer architecture for time series forecasting, aiming at overcoming the time delay and accumulative error issues in the canonical Transformer. Moreover, we present a novel spatial-temporal attention mechanism, building a bridge by a learned temporal influence map to fill the gaps between the spatial and temporal attention, so that spatial and temporal dependencies can be processed integrally. Empirically, we evaluate our model on diversified ego-centric future localization datasets and demonstrate state-of-the-art performance on both real-time and accuracy.
arXiv commentNot a satisfying work and need to be reformed
References in corpus (12)
- Language Models are Few-Shot Learners
- LSTM Fully Convolutional Networks for Time Series Classification
- Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting
- Non-Autoregressive Neural Machine Translation
- Skeleton-based Action Recognition via Spatial and Temporal Transformer Networks
- Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case
- Spatial-Temporal Transformer Networks for Traffic Flow Forecasting
- Spatial Temporal Transformer Network for Skeleton-based Action Recognition
- SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving
- ConvTransformer: A Convolutional Transformer Network for Video Frame Synthesis
- Non-autoregressive Transformer by Position Learning
- Glancing Transformer for Non-Autoregressive Neural Machine Translation