Recognizing Predictive Substructures with Subgraph Information Bottleneck
arXiv:2103.11155
Abstract
The emergence of Graph Convolutional Network (GCN) has greatly boosted the progress of graph learning. However, two disturbing factors, noise and redundancy in graph data, and lack of interpretation for prediction results, impede further development of GCN. One solution is to recognize a predictive yet compressed subgraph to get rid of the noise and redundancy and obtain the interpretable part of the graph. This setting of subgraph is similar to the information bottleneck (IB) principle, which is less studied on graph-structured data and GCN. Inspired by the IB principle, we propose a novel subgraph information bottleneck (SIB) framework to recognize such subgraphs, named IB-subgraph. However, the intractability of mutual information and the discrete nature of graph data makes the objective of SIB notoriously hard to optimize. To this end, we introduce a bilevel optimization scheme coupled with a mutual information estimator for irregular graphs. Moreover, we propose a continuous relaxation for subgraph selection with a connectivity loss for stabilization. We further theoretically prove the error bound of our estimation scheme for mutual information and the noise-invariant nature of IB-subgraph. Extensive experiments on graph learning and large-scale point cloud tasks demonstrate the superior property of IB-subgraph.
arXiv admin note: substantial text overlap with arXiv:2010.05563
References in corpus (21)
- Categorical Reparameterization with Gumbel-Softmax
- Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning
- The information bottleneck method
- FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling
- Self-Attention Graph Pooling
- Adversarial Attacks on Graph Neural Networks via Meta Learning
- Deep Variational Information Bottleneck
- Learning Discrete Structures for Graph Neural Networks
- InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization
- DeepGCNs: Can GCNs Go as Deep as CNNs?
- Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud
- Multi-Objective Molecule Generation using Interpretable Substructures
- Edge Contraction Pooling for Graph Neural Networks
- Graph Information Bottleneck
- Subgraph Neural Networks
- InfoBot: Transfer and Exploration via the Information Bottleneck
- Unsupervised Speech Decomposition via Triple Information Bottleneck
- Weakly Supervised Semantic Point Cloud Segmentation:Towards 10X Fewer Labels
- Label-Efficient Point Cloud Semantic Segmentation: An Active Learning Approach
- Graph-Revised Convolutional Network
- Graph Information Bottleneck for Subgraph Recognition