Unsupervised Continual Learning Via Pseudo Labels
arXiv:2104.07164
Abstract
Continual learning aims to learn new tasks incrementally using less computation and memory resources instead of retraining the model from scratch whenever new task arrives. However, existing approaches are designed in supervised fashion assuming all data from new tasks have been manually annotated, which are not practical for many real-life applications. In this work, we propose to use pseudo label instead of the ground truth to make continual learning feasible in unsupervised mode. The pseudo labels of new data are obtained by applying global clustering algorithm and we propose to use the model updated from last incremental step as the feature extractor. Due to the scarcity of existing work, we introduce a new benchmark experimental protocol for unsupervised continual learning of image classification task under class-incremental setting where no class label is provided for each incremental learning step. Our method is evaluated on the CIFAR-100 and ImageNet (ILSVRC) datasets by incorporating the pseudo label with various existing supervised approaches and show promising results in unsupervised scenario.
arXiv commentAccepted paper for 2021 IJCAI, Workshop
References in corpus (13)
- Distilling the Knowledge in a Neural Network
- Deep Residual Learning for Image Recognition
- A continual learning survey: Defying forgetting in classification tasks
- On Tiny Episodic Memories in Continual Learning
- FearNet: Brain-Inspired Model for Incremental Learning
- Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
- Continual Learning with Deep Generative Replay
- Joint Unsupervised Learning of Deep Representations and Image Clusters
- Efficient Continual Learning in Neural Networks with Embedding Regularization
- A Strategy for an Uncompromising Incremental Learner
- Maintaining Discrimination and Fairness in Class Incremental Learning
- Incremental Learning In Online Scenario
- Few-Shot Unsupervised Continual Learning through Meta-Examples