GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs

Abstract

We propose a new network architecture, Gated Attention Networks (GaAN), for learning on graphs. Unlike the traditional multi-head attention mechanism, which equally consumes all attention heads, GaAN uses a convolutional sub-network to control each attention head's importance. We demonstrate the effectiveness of GaAN on the inductive node classification problem. Moreover, with GaAN as a building block, we construct the Graph Gated Recurrent Unit (GGRU) to address the traffic speed forecasting problem. Extensive experiments on three real-world datasets show that our GaAN framework achieves state-of-the-art results on both tasks.

References in corpus (11)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Inductive Representation Learning on Large Graphs
• Spectral Networks and Locally Connected Networks on Graphs
• Convolutional Sequence to Sequence Learning
• Empirical Evaluation of Rectified Activations in Convolutional Network
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• A Structured Self-attentive Sentence Embedding
• Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model
• FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results

Cited by in corpus (73)

• Graph Neural Networks: A Review of Methods and Applications
• Graph Neural Network for Traffic Forecasting: A Survey
• Deep Learning on Traffic Prediction: Methods, Analysis and Future Directions
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• Deep Graph Contrastive Representation Learning
• GraphSAINT: Graph Sampling Based Inductive Learning Method
• DeeperGCN: All You Need to Train Deeper GCNs
• Traffic Prediction using Artificial Intelligence: Review of Recent Advances and Emerging Opportunities
• DeepGCNs: Making GCNs Go as Deep as CNNs
• ETA Prediction with Graph Neural Networks in Google Maps
• Transferring Robustness for Graph Neural Network Against Poisoning Attacks
• ST-GRAT: A Novel Spatio-temporal Graph Attention Network for Accurately Forecasting Dynamically Changing Road Speed
• Relational Graph Attention Networks
• IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control
• Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification
• Unifying Graph Convolutional Neural Networks and Label Propagation
• Graph Information Bottleneck
• Temporal Graph Networks for Deep Learning on Dynamic Graphs
• Revisiting Spatial-Temporal Similarity: A Deep Learning Framework for Traffic Prediction
• Machine Learning for Spatiotemporal Sequence Forecasting: A Survey
• Graph WaveNet for Deep Spatial-Temporal Graph Modeling
• Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecasting
• Understanding Attention and Generalization in Graph Neural Networks
• Graph-MLP: Node Classification without Message Passing in Graph
• Graph Representation Learning via Graphical Mutual Information Maximization
• CDSA: Cross-Dimensional Self-Attention for Multivariate, Geo-tagged Time Series Imputation
• Learning Graph Neural Networks with Positive and Unlabeled Nodes
• Multi-hop Attention Graph Neural Network
• Graph Star Net for Generalized Multi-Task Learning
• Graph Neural Networks: Methods, Applications, and Opportunities
• GraphCL: Contrastive Self-Supervised Learning of Graph Representations
• SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks
• Improving Graph Attention Networks with Large Margin-based Constraints
• Bayesian graph convolutional neural networks via tempered MCMC
• Residual or Gate? Towards Deeper Graph Neural Networks for Inductive Graph Representation Learning
• Learned Low Precision Graph Neural Networks
• Spatio-Temporal Deep Graph Infomax
• NODE-SELECT: A Graph Neural Network Based On A Selective Propagation Technique
• Fast Graph Attention Networks Using Effective Resistance Based Graph Sparsification
• Visual Relationship Detection using Scene Graphs: A Survey
• SPAGAN: Shortest Path Graph Attention Network
• Adversarially Regularized Graph Attention Networks for Inductive Learning on Partially Labeled Graphs
• The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learning
• Rubik: A Hierarchical Architecture for Efficient Graph Learning
• The Impact of Global Structural Information in Graph Neural Networks Applications
• Learning from Multiple Cities: A Meta-Learning Approach for Spatial-Temporal Prediction
• Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension
• Graph Signal Processing -- Part III: Machine Learning on Graphs, from Graph Topology to Applications
• Graph Convolution with Low-rank Learnable Local Filters
• Conditionally Parameterized, Discretization-Aware Neural Networks for Mesh-Based Modeling of Physical Systems
• Graph Attention Recurrent Neural Networks for Correlated Time Series Forecasting -- Full version
• Dynamic Spatiotemporal Graph Neural Network with Tensor Network
• GACAN: Graph Attention-Convolution-Attention Networks for Traffic Forecasting Based on Multi-granularity Time Series
• Dimensional Reweighting Graph Convolutional Networks
• Structure-Aware Label Smoothing for Graph Neural Networks
• TSAM: Temporal Link Prediction in Directed Networks based on Self-Attention Mechanism
• A Comparative Study on Basic Elements of Deep Learning Models for Spatial-Temporal Traffic Forecasting
• Improving Conversational Recommendation System by Pretraining on Billions Scale of Knowledge Graph
• Learning Robust Representations with Graph Denoising Policy Network
• Collective Link Prediction Oriented Network Embedding with Hierarchical Graph Attention
• GAIN: Graph Attention & Interaction Network for Inductive Semi-Supervised Learning over Large-scale Graphs
• Non-Recursive Graph Convolutional Networks
• Graph Feature Gating Networks
• Label-Aware Graph Convolutional Networks
• CoRGi: Content-Rich Graph Neural Networks with Attention
• Action Recognition with Kernel-based Graph Convolutional Networks
• Passenger Mobility Prediction via Representation Learning for Dynamic Directed and Weighted Graph
• Joint Forecasting and Interpolation of Graph Signals Using Deep Learning
• Supervised learning on heterogeneous, attributed entities interacting over time
• On the Global Self-attention Mechanism for Graph Convolutional Networks
• Decoding Spiking Mechanism with Dynamic Learning on Neuron Population
• FeatureNorm: L2 Feature Normalization for Dynamic Graph Embedding
• Learning Chebyshev Basis in Graph Convolutional Networks for Skeleton-based Action Recognition