InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization

Abstract

This paper studies learning the representations of whole graphs in both unsupervised and semi-supervised scenarios. Graph-level representations are critical in a variety of real-world applications such as predicting the properties of molecules and community analysis in social networks. Traditional graph kernel based methods are simple, yet effective for obtaining fixed-length representations for graphs but they suffer from poor generalization due to hand-crafted designs. There are also some recent methods based on language models (e.g. graph2vec) but they tend to only consider certain substructures (e.g. subtrees) as graph representatives. Inspired by recent progress of unsupervised representation learning, in this paper we proposed a novel method called InfoGraph for learning graph-level representations. We maximize the mutual information between the graph-level representation and the representations of substructures of different scales (e.g., nodes, edges, triangles). By doing so, the graph-level representations encode aspects of the data that are shared across different scales of substructures. Furthermore, we further propose InfoGraph*, an extension of InfoGraph for semi-supervised scenarios. InfoGraph* maximizes the mutual information between unsupervised graph representations learned by InfoGraph and the representations learned by existing supervised methods. As a result, the supervised encoder learns from unlabeled data while preserving the latent semantic space favored by the current supervised task. Experimental results on the tasks of graph classification and molecular property prediction show that InfoGraph is superior to state-of-the-art baselines and InfoGraph* can achieve performance competitive with state-of-the-art semi-supervised models.

ICLR 2020 (spotlight)

References in corpus (28)

• Adam: A Method for Stochastic Optimization
• Efficient Estimation of Word Representations in Vector Space
• Distributed Representations of Words and Phrases and their Compositionality
• Finding and evaluating community structure in networks
• Semi-Supervised Classification with Graph Convolutional Networks
• Distributed Representations of Sentences and Documents
• Neural Message Passing for Quantum Chemistry
• Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties
• Learning deep representations by mutual information estimation and maximization
• Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Temporal Ensembling for Semi-Supervised Learning
• Fast Graph Representation Learning with PyTorch Geometric
• Representation Learning on Graphs with Jumping Knowledge Networks
• Interpolation Consistency Training for Semi-Supervised Learning
• Biological network comparison using graphlet degree distribution
• Junction Tree Variational Autoencoder for Molecular Graph Generation
• Disentangling by Factorising
• graph2vec: Learning Distributed Representations of Graphs
• Manifold Mixup: Better Representations by Interpolating Hidden States
• How Powerful are Graph Neural Networks?
• Adversarial Examples for Evaluating Reading Comprehension Systems
• Neural Relational Inference for Interacting Systems
• Anonymous Walk Embeddings
• On Valid Optimal Assignment Kernels and Applications to Graph Classification
• The Multiscale Laplacian Graph Kernel
• Utilizing Edge Features in Graph Neural Networks via Variational Information Maximization
• Semi-supervised learning of hierarchical representations of molecules using neural message passing

Cited by in corpus (35)

• Graph Contrastive Learning with Augmentations
• Self-Supervised Graph Transformer on Large-Scale Molecular Data
• A Comprehensive Survey on Deep Graph Representation Learning
• Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
• Multi-view Graph Contrastive Representation Learning for Drug-Drug Interaction Prediction
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• Graph Information Bottleneck
• Heterogeneous Deep Graph Infomax
• Motif-based Graph Self-Supervised Learning for Molecular Property Prediction
• InfoGCL: Information-Aware Graph Contrastive Learning
• Graph Neural Networks: Methods, Applications, and Opportunities
• Heterogeneous Network Representation Learning: A Unified Framework with Survey and Benchmark
• TCL: Transformer-based Dynamic Graph Modelling via Contrastive Learning
• Learn molecular representations from large-scale unlabeled molecules for drug discovery
• GraphCrop: Subgraph Cropping for Graph Classification
• Motif-Driven Contrastive Learning of Graph Representations
• Fairness-Aware Node Representation Learning
• Iterative Graph Self-Distillation
• Self-Supervised Graph Co-Training for Session-based Recommendation
• Recognizing Predictive Substructures with Subgraph Information Bottleneck
• CogDL: A Comprehensive Library for Graph Deep Learning
• Self-supervised Graph Representation Learning via Bootstrapping
• Maximum Entropy Weighted Independent Set Pooling for Graph Neural Networks
• Improving Molecular Design by Stochastic Iterative Target Augmentation
• Label Contrastive Coding based Graph Neural Network for Graph Classification
• Leveraging Meta-path Contexts for Classification in Heterogeneous Information Networks
• Permutation-Invariant Variational Autoencoder for Graph-Level Representation Learning
• CGCL: Collaborative Graph Contrastive Learning without Handcrafted Graph Data Augmentations
• Graph Self-Supervised Learning with Learnable Structural and Positional Encodings
• Probing Negative Sampling Strategies to Learn GraphRepresentations via Unsupervised Contrastive Learning
• Data-Driven Self-Supervised Graph Representation Learning
• Self-Supervised Graph Learning with Proximity-based Views and Channel Contrast
• Towards Self-Explainable Graph Neural Network
• RRLFSOR: An Efficient Self-Supervised Learning Strategy of Graph Convolutional Networks
• Pairwise Half-graph Discrimination: A Simple Graph-level Self-supervised Strategy for Pre-training Graph Neural Networks