Automatically Discovering and Learning New Visual Categories with Ranking Statistics

Abstract

We tackle the problem of discovering novel classes in an image collection given labelled examples of other classes. This setting is similar to semi-supervised learning, but significantly harder because there are no labelled examples for the new classes. The challenge, then, is to leverage the information contained in the labelled images in order to learn a general-purpose clustering model and use the latter to identify the new classes in the unlabelled data. In this work we address this problem by combining three ideas: (1) we suggest that the common approach of bootstrapping an image representation using the labeled data only introduces an unwanted bias, and that this can be avoided by using self-supervised learning to train the representation from scratch on the union of labelled and unlabelled data; (2) we use rank statistics to transfer the model's knowledge of the labelled classes to the problem of clustering the unlabelled images; and, (3) we train the data representation by optimizing a joint objective function on the labelled and unlabelled subsets of the data, improving both the supervised classification of the labelled data, and the clustering of the unlabelled data. We evaluate our approach on standard classification benchmarks and outperform current methods for novel category discovery by a significant margin.

ICLR 2020, code: http://www.robots.ox.ac.uk/~vgg/research/auto_novel

References in corpus (6)

• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering
• Semi-Supervised Learning with Ladder Networks
• Joint Unsupervised Learning of Deep Representations and Image Clusters
• Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization
• Deep Image Category Discovery using a Transferred Similarity Function

Cited by in corpus (9)

• AutoNovel: Automatically Discovering and Learning Novel Visual Categories
• Open-World Semi-Supervised Learning
• Open-Set Recognition: a Good Closed-Set Classifier is All You Need?
• Novel Visual Category Discovery with Dual Ranking Statistics and Mutual Knowledge Distillation
• Continuum: Simple Management of Complex Continual Learning Scenarios
• Self-Supervised Features Improve Open-World Learning
• Refining Pseudo Labels with Clustering Consensus over Generations for Unsupervised Object Re-identification
• Joint Representation Learning and Novel Category Discovery on Single- and Multi-modal Data
• Open-Set Representation Learning through Combinatorial Embedding