CUCL: Codebook for Unsupervised Continual Learning

Abstract

The focus of this study is on Unsupervised Continual Learning (UCL), as it presents an alternative to Supervised Continual Learning which needs high-quality manual labeled data. The experiments under the UCL paradigm indicate a phenomenon where the results on the first few tasks are suboptimal. This phenomenon can render the model inappropriate for practical applications. To address this issue, after analyzing the phenomenon and identifying the lack of diversity as a vital factor, we propose a method named Codebook for Unsupervised Continual Learning (CUCL) which promotes the model to learn discriminative features to complete the class boundary. Specifically, we first introduce a Product Quantization to inject diversity into the representation and apply a cross quantized contrastive loss between the original representation and the quantized one to capture discriminative information. Then, based on the quantizer, we propose an effective Codebook Rehearsal to address catastrophic forgetting. This study involves conducting extensive experiments on CIFAR100, TinyImageNet, and MiniImageNet benchmark datasets. Our method significantly boosts the performances of supervised and unsupervised methods. For instance, on TinyImageNet, our method led to a relative improvement of 12.76% and 7% when compared with Simsiam and BYOL, respectively.

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

References in corpus (14)

• Overcoming catastrophic forgetting in neural networks
• A Simple Framework for Contrastive Learning of Visual Representations
• Bootstrap your own latent: A new approach to self-supervised Learning
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Learning deep representations by mutual information estimation and maximization
• Progressive Neural Networks
• Continual Learning Through Synaptic Intelligence
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• Efficient Lifelong Learning with A-GEM
• Overcoming catastrophic forgetting with hard attention to the task
• Efficient Continual Learning with Modular Networks and Task-Driven Priors
• Representational Continuity for Unsupervised Continual Learning
• EEC: Learning to Encode and Regenerate Images for Continual Learning
• Class Gradient Projection For Continual Learning