Graph Information Bottleneck

Abstract

Representation learning of graph-structured data is challenging because both graph structure and node features carry important information. Graph Neural Networks (GNNs) provide an expressive way to fuse information from network structure and node features. However, GNNs are prone to adversarial attacks. Here we introduce Graph Information Bottleneck (GIB), an information-theoretic principle that optimally balances expressiveness and robustness of the learned representation of graph-structured data. Inheriting from the general Information Bottleneck (IB), GIB aims to learn the minimal sufficient representation for a given task by maximizing the mutual information between the representation and the target, and simultaneously constraining the mutual information between the representation and the input data. Different from the general IB, GIB regularizes the structural as well as the feature information. We design two sampling algorithms for structural regularization and instantiate the GIB principle with two new models: GIB-Cat and GIB-Bern, and demonstrate the benefits by evaluating the resilience to adversarial attacks. We show that our proposed models are more robust than state-of-the-art graph defense models. GIB-based models empirically achieve up to 31% improvement with adversarial perturbation of the graph structure as well as node features.

arXiv comment20 pages, 3 figures, NeurIPS 2020

References in corpus (26)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Semi-Supervised Classification with Graph Convolutional Networks
• Neural Message Passing for Quantum Chemistry
• Categorical Reparameterization with Gumbel-Softmax
• Spatial Networks
• The information bottleneck method
• Learning deep representations by mutual information estimation and maximization
• Fast Graph Representation Learning with PyTorch Geometric
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Adversarial Attacks on Neural Networks for Graph Data
• Representation Learning on Graphs with Jumping Knowledge Networks
• Variational Graph Auto-Encoders
• Estimating divergence functionals and the likelihood ratio by convex risk minimization
• Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
• Deep Variational Information Bottleneck
• GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs
• Adversarial Attack on Graph Structured Data
• On Variational Bounds of Mutual Information
• InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization
• Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow
• Geom-GCN: Geometric Graph Convolutional Networks
• Adversarial Examples on Graph Data: Deep Insights into Attack and Defense
• The Conditional Entropy Bottleneck
• Position-aware Graph Neural Networks
• Measuring and Improving the Use of Graph Information in Graph Neural Networks
• Optimizing Generalized PageRank Methods for Seed-Expansion Community Detection

Cited by in corpus (12)

• Earthquake Phase Association with Graph Neural Networks
• SE-GSL: A General and Effective Graph Structure Learning Framework through Structural Entropy Optimization
• Adversarial Attack Framework on Graph Embedding Models with Limited Knowledge
• MixupExplainer: Generalizing Explanations for Graph Neural Networks with Data Augmentation
• GraphGLOW: Universal and Generalizable Structure Learning for Graph Neural Networks
• Unleashing the Power of Large Language Model for Denoising Recommendation
• PAC-Bayes Information Bottleneck
• Learning Robust Representation through Graph Adversarial Contrastive Learning
• Recognizing Predictive Substructures with Subgraph Information Bottleneck
• Neuron Campaign for Initialization Guided by Information Bottleneck Theory
• Graph Information Bottleneck for Remote Sensing Segmentation
• Is Your Explanation Reliable: Confidence-Aware Explanation on Graph Neural Networks