Temporal Graph Networks for Deep Learning on Dynamic Graphs

Abstract

Graph Neural Networks (GNNs) have recently become increasingly popular due to their ability to learn complex systems of relations or interactions arising in a broad spectrum of problems ranging from biology and particle physics to social networks and recommendation systems. Despite the plethora of different models for deep learning on graphs, few approaches have been proposed thus far for dealing with graphs that present some sort of dynamic nature (e.g. evolving features or connectivity over time). In this paper, we present Temporal Graph Networks (TGNs), a generic, efficient framework for deep learning on dynamic graphs represented as sequences of timed events. Thanks to a novel combination of memory modules and graph-based operators, TGNs are able to significantly outperform previous approaches being at the same time more computationally efficient. We furthermore show that several previous models for learning on dynamic graphs can be cast as specific instances of our framework. We perform a detailed ablation study of different components of our framework and devise the best configuration that achieves state-of-the-art performance on several transductive and inductive prediction tasks for dynamic graphs.

References in corpus (24)

• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Geometric deep learning: going beyond Euclidean data
• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• Modeling polypharmacy side effects with graph convolutional networks
• Variational Graph Auto-Encoders
• Disease Prediction using Graph Convolutional Networks: Application to Autism Spectrum Disorder and Alzheimer's Disease
• Interaction Networks for Learning about Objects, Relations and Physics
• Dynamic Graph Convolutional Networks
• Representation Learning on Graphs: Methods and Applications
• DynGEM: Deep Embedding Method for Dynamic Graphs
• Link Prediction Based on Graph Neural Networks
• GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs
• Fake News Detection on Social Media using Geometric Deep Learning
• SIGN: Scalable Inception Graph Neural Networks
• Node Embedding over Temporal Graphs
• GC-LSTM: Graph Convolution Embedded LSTM for Dynamic Link Prediction
• Learning Human-Object Interactions by Graph Parsing Neural Networks
• Time2Vec: Learning a Vector Representation of Time
• 3D Graph Convolutional Networks with Temporal Graphs: A Spatial Information Free Framework For Traffic Forecasting
• Geometric deep learning on graphs and manifolds using mixture model CNNs
• Temporal Knowledge Graph Embedding Model based on Additive Time Series Decomposition
• Streaming Graph Neural Networks
• Privacy-Aware Recommender Systems Challenge on Twitter's Home Timeline

Cited by in corpus (29)

• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A survey
• Graph Representation Learning in Biomedicine
• ETA Prediction with Graph Neural Networks in Google Maps
• Combinatorial Optimization with Physics-Inspired Graph Neural Networks
• Learning Dynamic Graph Representation of Brain Connectome with Spatio-Temporal Attention
• The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Survey
• Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks
• Efficient Scaling of Dynamic Graph Neural Networks
• Lifelong Learning of Graph Neural Networks for Open-World Node Classification
• TCL: Transformer-based Dynamic Graph Modelling via Contrastive Learning
• Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs
• On the Equivalence Between Temporal and Static Graph Representations for Observational Predictions
• Network In Graph Neural Network
• Link Prediction using Graph Neural Networks for Master Data Management
• Utilising Graph Machine Learning within Drug Discovery and Development
• Event2Graph: Event-driven Bipartite Graph for Multivariate Time-series Anomaly Detection
• DyFormer: A Scalable Dynamic Graph Transformer with Provable Benefits on Generalization Ability
• baller2vec++: A Look-Ahead Multi-Entity Transformer For Modeling Coordinated Agents
• Persistent Message Passing
• Financial Crime & Fraud Detection Using Graph Computing: Application Considerations & Outlook
• AdaGNN: A multi-modal latent representation meta-learner for GNNs based on AdaBoosting
• Learning Representation over Dynamic Graph using Aggregation-Diffusion Mechanism
• Graph Computing for Financial Crime and Fraud Detection: Trends, Challenges and Outlook
• ConTIG: Continuous Representation Learning on Temporal Interaction Graphs
• Graph Neural Networks in Real-Time Fraud Detection with Lambda Architecture
• Understanding Higher-order Structures in Evolving Graphs: A Simplicial Complex based Kernel Estimation Approach
• Temporal Graph Network Embedding with Causal Anonymous Walks Representations
• Spectral Transform Forms Scalable Transformer