CLDG: Contrastive Learning on Dynamic Graphs

Abstract

The graph with complex annotations is the most potent data type, whose constantly evolving motivates further exploration of the unsupervised dynamic graph representation. One of the representative paradigms is graph contrastive learning. It constructs self-supervised signals by maximizing the mutual information between the statistic graph's augmentation views. However, the semantics and labels may change within the augmentation process, causing a significant performance drop in downstream tasks. This drawback becomes greatly magnified on dynamic graphs. To address this problem, we designed a simple yet effective framework named CLDG. Firstly, we elaborate that dynamic graphs have temporal translation invariance at different levels. Then, we proposed a sampling layer to extract the temporally-persistent signals. It will encourage the node to maintain consistent local and global representations, i.e., temporal translation invariance under the timespan views. The extensive experiments demonstrate the effectiveness and efficiency of the method on seven datasets by outperforming eight unsupervised state-of-the-art baselines and showing competitiveness against four semi-supervised methods. Compared with the existing dynamic graph method, the number of model parameters and training time is reduced by an average of 2,001.86 times and 130.31 times on seven datasets, respectively.

Accepted by ICDE2023

References in corpus (23)

• Adam: A Method for Stochastic Optimization
• DeepWalk: Online Learning of Social Representations
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• LINE: Large-scale Information Network Embedding
• Bootstrap your own latent: A new approach to self-supervised Learning
• Learning deep representations by mutual information estimation and maximization
• Graph Contrastive Learning with Adaptive Augmentation
• Graph Contrastive Learning with Augmentations
• Hierarchical Graph Representation Learning with Differentiable Pooling
• Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks
• dyngraph2vec: Capturing Network Dynamics using Dynamic Graph Representation Learning
• Deep Graph Contrastive Representation Learning
• Contrastive Multi-View Representation Learning on Graphs
• Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search
• SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation
• DynGEM: Deep Embedding Method for Dynamic Graphs
• ETA Prediction with Graph Neural Networks in Google Maps
• Diffusion Improves Graph Learning
• Chip Placement with Deep Reinforcement Learning
• Combining Label Propagation and Simple Models Out-performs Graph Neural Networks
• Graph Neural Networks in Recommender Systems: A Survey
• Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks