Spatial-Temporal Transformer Networks for Traffic Flow Forecasting

Abstract

Traffic forecasting has emerged as a core component of intelligent transportation systems. However, timely accurate traffic forecasting, especially long-term forecasting, still remains an open challenge due to the highly nonlinear and dynamic spatial-temporal dependencies of traffic flows. In this paper, we propose a novel paradigm of Spatial-Temporal Transformer Networks (STTNs) that leverages dynamical directed spatial dependencies and long-range temporal dependencies to improve the accuracy of long-term traffic forecasting. Specifically, we present a new variant of graph neural networks, named spatial transformer, by dynamically modeling directed spatial dependencies with self-attention mechanism to capture realtime traffic conditions as well as the directionality of traffic flows. Furthermore, different spatial dependency patterns can be jointly modeled with multi-heads attention mechanism to consider diverse relationships related to different factors (e.g. similarity, connectivity and covariance). On the other hand, the temporal transformer is utilized to model long-range bidirectional temporal dependencies across multiple time steps. Finally, they are composed as a block to jointly model the spatial-temporal dependencies for accurate traffic prediction. Compared to existing works, the proposed model enables fast and scalable training over a long range spatial-temporal dependencies. Experiment results demonstrate that the proposed model achieves competitive results compared with the state-of-the-arts, especially forecasting long-term traffic flows on real-world PeMS-Bay and PeMSD7(M) datasets.

References in corpus (6)

• Sequence to Sequence Learning with Neural Networks
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Semi-Supervised Classification with Graph Convolutional Networks
• Neural Message Passing for Quantum Chemistry
• Convolutional Sequence to Sequence Learning
• Short-term traffic flow forecasting with spatial-temporal correlation in a hybrid deep learning framework

Cited by in corpus (14)

• Graph Neural Network for Traffic Forecasting: A Survey
• Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting
• LSTTN: A Long-Short Term Transformer-based Spatio-temporal Neural Network for Traffic Flow Forecasting
• Long-Range Transformers for Dynamic Spatiotemporal Forecasting
• Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecasting
• Spatial-temporal Conv-sequence Learning with Accident Encoding for Traffic Flow Prediction
• NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting
• Periodic Residual Learning for Crowd Flow Forecasting
• MemDA: Forecasting Urban Time Series with Memory-based Drift Adaptation
• MAF-GNN: Multi-adaptive Spatiotemporal-flow Graph Neural Network for Traffic Speed Forecasting
• Auto-STGCN: Autonomous Spatial-Temporal Graph Convolutional Network Search Based on Reinforcement Learning and Existing Research Results
• GroupFormer: Group Activity Recognition with Clustered Spatial-Temporal Transformer
• Transformers à Grande Vitesse
• On the Global Self-attention Mechanism for Graph Convolutional Networks