Unsupervised Continual Learning Via Pseudo Labels
arXiv:2104.07164
Abstract
Continual learning aims to learn new tasks incrementally using less computation and memory resources instead of retraining the model from scratch whenever new task arrives. However, existing approaches are designed in supervised fashion assuming all data from new tasks have been manually annotated, which are not practical for many real-life applications. In this work, we propose to use pseudo label instead of the ground truth to make continual learning feasible in unsupervised mode. The pseudo labels of new data are obtained by applying global clustering algorithm and we propose to use the model updated from last incremental step as the feature extractor. Due to the scarcity of existing work, we introduce a new benchmark experimental protocol for unsupervised continual learning of image classification task under class-incremental setting where no class label is provided for each incremental learning step. Our method is evaluated on the CIFAR-100 and ImageNet (ILSVRC) datasets by incorporating the pseudo label with various existing supervised approaches and show promising results in unsupervised scenario.
Accepted paper for 2021 IJCAI, Workshop
References in corpus (8)
- Distilling the Knowledge in a Neural Network
- A continual learning survey: Defying forgetting in classification tasks
- On Tiny Episodic Memories in Continual Learning
- Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
- Efficient Continual Learning in Neural Networks with Embedding Regularization
- Maintaining Discrimination and Fairness in Class Incremental Learning
- Incremental Learning In Online Scenario
- Few-Shot Unsupervised Continual Learning through Meta-Examples