Domain-Aware Augmentations for Unsupervised Online General Continual Learning

Abstract

Continual Learning has been challenging, especially when dealing with unsupervised scenarios such as Unsupervised Online General Continual Learning (UOGCL), where the learning agent has no prior knowledge of class boundaries or task change information. While previous research has focused on reducing forgetting in supervised setups, recent studies have shown that self-supervised learners are more resilient to forgetting. This paper proposes a novel approach that enhances memory usage for contrastive learning in UOGCL by defining and using stream-dependent data augmentations together with some implementation tricks. Our proposed method is simple yet effective, achieves state-of-the-art results compared to other unsupervised approaches in all considered setups, and reduces the gap between supervised and unsupervised continual learning. Our domain-aware augmentation procedure can be adapted to other replay-based methods, making it a promising strategy for continual learning.

arXiv commentAccepted to BMVC'23

References in corpus (9)

• SMOTE: Synthetic Minority Over-sampling Technique
• A Simple Framework for Contrastive Learning of Visual Representations
• mixup: Beyond Empirical Risk Minimization
• Bootstrap your own latent: A new approach to self-supervised Learning
• Experience Replay for Continual Learning
• Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
• New Insights on Reducing Abrupt Representation Change in Online Continual Learning
• Representational Continuity for Unsupervised Continual Learning
• New Insights on Relieving Task-Recency Bias for Online Class Incremental Learning