Graph Convolution for Semi-Supervised Classification: Improved Linear Separability and Out-of-Distribution Generalization
arXiv:2102.06966
Abstract
Recently there has been increased interest in semi-supervised classification in the presence of graphical information. A new class of learning models has emerged that relies, at its most basic level, on classifying the data after first applying a graph convolution. To understand the merits of this approach, we study the classification of a mixture of Gaussians, where the data corresponds to the node attributes of a stochastic block model. We show that graph convolution extends the regime in which the data is linearly separable by a factor of roughly , where is the expected degree of a node, as compared to the mixture model data on its own. Furthermore, we find that the linear classifier obtained by minimizing the cross-entropy loss after the graph convolution generalizes to out-of-distribution data where the unseen data can have different intra- and inter-class edge probabilities from the training data.
arXiv comment30 pages, 9 figures, 2 tables
References in corpus (13)
- Inductive Representation Learning on Large Graphs
- Neural Message Passing for Quantum Chemistry
- Fast Graph Representation Learning with PyTorch Geometric
- Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications
- Interaction Networks for Learning about Objects, Relations and Physics
- Covariate-assisted spectral clustering
- Generalization and Representational Limits of Graph Neural Networks
- Information-theoretic thresholds for community detection in sparse networks
- Asymptotic Mutual Information for the Two-Groups Stochastic Block Model
- Contextual Stochastic Block Models
- Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks
- Stochastic Blockmodels meet Graph Neural Networks
- How hard is to distinguish graphs with graph neural networks?