AutoNovel: Automatically Discovering and Learning Novel Visual Categories

Abstract

We tackle the problem of discovering novel classes in an image collection given labelled examples of other classes. We present a new approach called AutoNovel to address this problem by combining three ideas: (1) we suggest that the common approach of bootstrapping an image representation using the labelled data only introduces an unwanted bias, and that this can be avoided by using self-supervised learning to train the representation from scratch on the union of labelled and unlabelled data; (2) we use ranking statistics to transfer the model's knowledge of the labelled classes to the problem of clustering the unlabelled images; and, (3) we train the data representation by optimizing a joint objective function on the labelled and unlabelled subsets of the data, improving both the supervised classification of the labelled data, and the clustering of the unlabelled data. Moreover, we propose a method to estimate the number of classes for the case where the number of new categories is not known a priori. We evaluate AutoNovel on standard classification benchmarks and substantially outperform current methods for novel category discovery. In addition, we also show that AutoNovel can be used for fully unsupervised image clustering, achieving promising results.

arXiv commentTPAMI 2021, code: http://www.robots.ox.ac.uk/~vgg/research/auto_novel/. arXiv admin note: substantial text overlap with arXiv:2002.05714

References in corpus (6)

• Improved Baselines with Momentum Contrastive Learning
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Unsupervised Representation Learning by Predicting Image Rotations
• Semi-Supervised Learning with Ladder Networks
• Automatically Discovering and Learning New Visual Categories with Ranking Statistics
• Deep Image Category Discovery using a Transferred Similarity Function

Cited by in corpus (10)

• Learning Representation for Clustering via Prototype Scattering and Positive Sampling
• Open-Set Recognition: a Good Closed-Set Classifier is All You Need?
• A Clustering Framework for Unsupervised and Semi-supervised New Intent Discovery
• ProtoGCD: Unified and Unbiased Prototype Learning for Generalized Category Discovery
• WOT-Class: Weakly Supervised Open-world Text Classification
• Joint Representation Learning and Novel Category Discovery on Single- and Multi-modal Data
• A Practical Approach to Novel Class Discovery in Tabular Data
• A Method for Discovering Novel Classes in Tabular Data
• Long-Tailed Learning for Generalized Category Discovery
• Exploring the Tradeoff Between Diversity and Discrimination for Continuous Category Discovery