Few-Shot Unsupervised Continual Learning through Meta-Examples
arXiv:2009.08107
Abstract
In real-world applications, data do not reflect the ones commonly used for neural networks training, since they are usually few, unlabeled and can be available as a stream. Hence many existing deep learning solutions suffer from a limited range of applications, in particular in the case of online streaming data that evolve over time. To narrow this gap, in this work we introduce a novel and complex setting involving unsupervised meta-continual learning with unbalanced tasks. These tasks are built through a clustering procedure applied to a fitted embedding space. We exploit a meta-learning scheme that simultaneously alleviates catastrophic forgetting and favors the generalization to new tasks. Moreover, to encourage feature reuse during the meta-optimization, we exploit a single inner loop taking advantage of an aggregated representation achieved through the use of a self-attention mechanism. Experimental results on few-shot learning benchmarks show competitive performance even compared to the supervised case. Additionally, we empirically observe that in an unsupervised scenario, the small tasks and the variability in the clusters pooling play a crucial role in the generalization capability of the network. Further, on complex datasets, the exploitation of more clusters than the true number of classes leads to higher results, even compared to the ones obtained with full supervision, suggesting that a predefined partitioning into classes can miss relevant structural information.
Accepted at 34th Conference on Neural Information Processing Systems (NeurIPS 2020), 4th Workshop on Meta-Learning, 16 pages, 3 figures
References in corpus (17)
- Overcoming catastrophic forgetting in neural networks
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Continual Learning Through Synaptic Intelligence
- Progress & Compress: A scalable framework for continual learning
- Meta-Learning with Implicit Gradients
- Fast Context Adaptation via Meta-Learning
- Unsupervised Learning via Meta-Learning
- Hierarchically Structured Meta-learning
- Online Meta-Learning
- Continual Unsupervised Representation Learning
- Unsupervised Meta-Learning For Few-Shot Image Classification
- Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer
- Defining Benchmarks for Continual Few-Shot Learning
- Meta Continual Learning
- Unsupervised Few-shot Learning via Self-supervised Training
- Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning
- Meta-learnt priors slow down catastrophic forgetting in neural networks