CycleCluster: Modernising Clustering Regularisation for Deep Semi-Supervised Classification

Abstract

Given the potential difficulties in obtaining large quantities of labelled data, many works have explored the use of deep semi-supervised learning, which uses both labelled and unlabelled data to train a neural network architecture. The vast majority of SSL approaches focus on implementing the low-density separation assumption or consistency assumption, the idea that decision boundaries should lie in low density regions. However, they have implemented this assumption by making local changes to the decision boundary at each data point, ignoring the global structure of the data. In this work, we explore an alternative approach using the global information present in the clustered data to update our decision boundaries. We propose a novel framework, CycleCluster, for deep semi-supervised classification. Our core optimisation is driven by a new clustering based regularisation along with a graph based pseudo-labels and a shared deep network. Demonstrating that direct implementation of the cluster assumption is a viable alternative to the popular consistency based regularisation. We demonstrate the predictive capability of our technique through a careful set of numerical results.

References in corpus (6)

• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Interpolation Consistency Training for Semi-Supervised Learning
• Unsupervised Learning by Predicting Noise
• Adversarial Robustness May Be at Odds With Simplicity
• Semi-Supervised Learning by Label Gradient Alignment
• Energy Models for Better Pseudo-Labels: Improving Semi-Supervised Classification with the 1-Laplacian Graph Energy