Every Node Counts: Self-Ensembling Graph Convolutional Networks for Semi-Supervised Learning
arXiv:1809.09925
Abstract
Graph convolutional network (GCN) provides a powerful means for graph-based semi-supervised tasks. However, as a localized first-order approximation of spectral graph convolution, the classic GCN can not take full advantage of unlabeled data, especially when the unlabeled node is far from labeled ones. To capitalize on the information from unlabeled nodes to boost the training for GCN, we propose a novel framework named Self-Ensembling GCN (SEGCN), which marries GCN with Mean Teacher - another powerful model in semi-supervised learning. SEGCN contains a student model and a teacher model. As a student, it not only learns to correctly classify the labeled nodes, but also tries to be consistent with the teacher on unlabeled nodes in more challenging situations, such as a high dropout rate and graph collapse. As a teacher, it averages the student model weights and generates more accurate predictions to lead the student. In such a mutual-promoting process, both labeled and unlabeled samples can be fully utilized for backpropagating effective gradients to train GCN. In three article classification tasks, i.e. Citeseer, Cora and Pubmed, we validate that the proposed method matches the state of the arts in the classification accuracy.
arXiv comment9 pages, 4 figures
References in corpus (14)
- Adam: A Method for Stochastic Optimization
- Distilling the Knowledge in a Neural Network
- Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
- Temporal Ensembling for Semi-Supervised Learning
- Random Erasing Data Augmentation
- Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning
- Shake-Shake regularization
- Dynamic Graph CNN for Learning on Point Clouds
- Deep Networks with Stochastic Depth
- Adaptive Graph Convolutional Neural Networks
- N-GCN: Multi-scale Graph Convolution for Semi-supervised Node Classification
- Self-ensembling for visual domain adaptation
- Bootstrapped Graph Diffusions: Exposing the Power of Nonlinearity
- Fusion Graph Convolutional Networks