Phase transitions and optimal algorithms for semi-supervised classifications on graphs: from belief propagation to graph convolution network

Abstract

We perform theoretical and algorithmic studies for the problem of clustering and semi-supervised classification on graphs with both pairwise relational information and single-point feature information, upon a joint stochastic block model for generating synthetic graphs with both edges and node features. Asymptotically exact analysis based on the Bayesian inference of the underlying model are conducted, using the cavity method in statistical physics. Theoretically, we identify a phase transition of the generative model, which puts fundamental limits on the ability of all possible algorithms in the clustering task of the underlying model. Algorithmically, we propose a belief propagation algorithm that is asymptotically optimal on the generative model, and can be further extended to a belief propagation graph convolution neural network (BPGCN) for semi-supervised classification on graphs. For the first time, well-controlled benchmark datasets with asymptotially exact properties and optimal solutions could be produced for the evaluation of graph convolution neural networks, and for the theoretical understanding of their strengths and weaknesses. In particular, on these synthetic benchmark networks we observe that existing graph convolution neural networks are subject to an sparsity issue and an ovefitting issue in practice, both of which are successfully overcome by our BPGCN. Moreover, when combined with classic neural network methods, BPGCN yields extraordinary classification performances on some real-world datasets that have never been achieved before.

arXiv comment18 pages, 21 figures

References in corpus (11)

• Cooperative Game Theory Approaches for Network Partitioning
• Comparing community structure identification
• Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications
• Spectral redemption: clustering sparse networks
• Scalable detection of statistically significant communities and hierarchies, using message-passing for modularity
• The Nishimori line and Bayesian Statistics
• Evaluating accuracy of community detection using the relative normalized mutual information
• Phase transitions in semisupervised clustering of sparse networks
• Robust reconstruction on trees is determined by the second eigenvalue
• Justice blocks and predictability of US Supreme Court votes
• Network structure, metadata and the prediction of missing nodes and annotations