Semi-Supervised Learning via Compact Latent Space Clustering

Abstract

We present a novel cost function for semi-supervised learning of neural networks that encourages compact clustering of the latent space to facilitate separation. The key idea is to dynamically create a graph over embeddings of labeled and unlabeled samples of a training batch to capture underlying structure in feature space, and use label propagation to estimate its high and low density regions. We then devise a cost function based on Markov chains on the graph that regularizes the latent space to form a single compact cluster per class, while avoiding to disturb existing clusters during optimization. We evaluate our approach on three benchmarks and compare to state-of-the art with promising results. Our approach combines the benefits of graph-based regularization with efficient, inductive inference, does not require modifications to a network architecture, and can thus be easily applied to existing networks to enable an effective use of unlabeled data.

arXiv commentPresented as a long oral in ICML 2018. Post-conference camera ready

Cited by in corpus (14)

• Domain Generalization via Optimal Transport with Metric Similarity Learning
• On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data
• Progressive Identification of True Labels for Partial-Label Learning
• Semi-Supervised Learning by Label Gradient Alignment
• Learning from Extrinsic and Intrinsic Supervisions for Domain Generalization
• Inversion dynamics of class manifolds in deep learning reveals tradeoffs underlying generalisation
• Intraclass clustering: an implicit learning ability that regularizes DNNs
• Consistency Regularization with Generative Adversarial Networks for Semi-Supervised Learning
• Local Clustering with Mean Teacher for Semi-supervised Learning
• Towards Robust Pattern Recognition: A Review
• Patch-level Neighborhood Interpolation: A General and Effective Graph-based Regularization Strategy
• Improving Distantly-supervised Entity Typing with Compact Latent Space Clustering
• End-To-End Graph-based Deep Semi-Supervised Learning
• Semi-supervised Learning with Contrastive Predicative Coding