Graph InfoClust: Leveraging cluster-level node information for unsupervised graph representation learning

Abstract

Unsupervised (or self-supervised) graph representation learning is essential to facilitate various graph data mining tasks when external supervision is unavailable. The challenge is to encode the information about the graph structure and the attributes associated with the nodes and edges into a low dimensional space. Most existing unsupervised methods promote similar representations across nodes that are topologically close. Recently, it was shown that leveraging additional graph-level information, e.g., information that is shared among all nodes, encourages the representations to be mindful of the global properties of the graph, which greatly improves their quality. However, in most graphs, there is significantly more structure that can be captured, e.g., nodes tend to belong to (multiple) clusters that represent structurally similar nodes. Motivated by this observation, we propose a graph representation learning method called Graph InfoClust (GIC), that seeks to additionally capture cluster-level information content. These clusters are computed by a differentiable K-means method and are jointly optimized by maximizing the mutual information between nodes of the same clusters. This optimization leads the node representations to capture richer information and nodal interactions, which improves their quality. Experiments show that GIC outperforms state-of-art methods in various downstream tasks (node classification, link prediction, and node clustering) with a 0.9% to 6.1% gain over the best competing approach, on average.

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Variational Graph Auto-Encoders
• Learning Convolutional Neural Networks for Graphs
• Pitfalls of Graph Neural Network Evaluation
• Diffusion-Convolutional Neural Networks
• HARP: Hierarchical Representation Learning for Networks

Cited by in corpus (14)

• Graph Self-Supervised Learning: A Survey
• Graph Neural Networks: Taxonomy, Advances and Trends
• Variational Graph Normalized Auto-Encoders
• A Unified Lottery Ticket Hypothesis for Graph Neural Networks
• USER: Unsupervised Structural Entropy-based Robust Graph Neural Network
• Neural Link Prediction with Walk Pooling
• Unsupervised Deep Manifold Attributed Graph Embedding
• Edge: Enriching Knowledge Graph Embeddings with External Text
• Learning Compact Representations of Neural Networks using DiscriminAtive Masking (DAM)
• HeMI: Multi-view Embedding in Heterogeneous Graphs
• Self-Supervised Deep Graph Embedding with High-Order Information Fusion for Community Discovery
• Barlow Graph Auto-Encoder for Unsupervised Network Embedding
• Memory-Associated Differential Learning
• GenURL: A General Framework for Unsupervised Representation Learning