Self-Supervised Learning for Large-Scale Unsupervised Image Clustering

Abstract

Unsupervised learning has always been appealing to machine learning researchers and practitioners, allowing them to avoid an expensive and complicated process of labeling the data. However, unsupervised learning of complex data is challenging, and even the best approaches show much weaker performance than their supervised counterparts. Self-supervised deep learning has become a strong instrument for representation learning in computer vision. However, those methods have not been evaluated in a fully unsupervised setting. In this paper, we propose a simple scheme for unsupervised classification based on self-supervised representations. We evaluate the proposed approach with several recent self-supervised methods showing that it achieves competitive results for ImageNet classification (39% accuracy on ImageNet with 1000 clusters and 46% with overclustering). We suggest adding the unsupervised evaluation to a set of standard benchmarks for self-supervised learning. The code is available at https://github.com/Randl/kmeans_selfsuper

arXiv commentaccepted to NeurIPS 2020 Workshop: Self-Supervised Learning - Theory and Practice

References in corpus (14)

• A Simple Framework for Contrastive Learning of Visual Representations
• Natural Scales in Geographical Patterns
• Bootstrap your own latent: A new approach to self-supervised Learning
• Improved Baselines with Momentum Contrastive Learning
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• Adversarial Feature Learning
• What Makes for Good Views for Contrastive Learning?
• Contrastive Multiview Coding
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• Large Scale Adversarial Representation Learning
• Self-labelling via simultaneous clustering and representation learning
• Are we done with ImageNet?
• Whitening for Self-Supervised Representation Learning