Wandering Within a World: Online Contextualized Few-Shot Learning

Abstract

We aim to bridge the gap between typical human and machine-learning environments by extending the standard framework of few-shot learning to an online, continual setting. In this setting, episodes do not have separate training and testing phases, and instead models are evaluated online while learning novel classes. As in the real world, where the presence of spatiotemporal context helps us retrieve learned skills in the past, our online few-shot learning setting also features an underlying context that changes throughout time. Object classes are correlated within a context and inferring the correct context can lead to better performance. Building upon this setting, we propose a new few-shot learning dataset based on large scale indoor imagery that mimics the visual experience of an agent wandering within a world. Furthermore, we convert popular few-shot learning approaches into online versions and we also propose a new contextual prototypical memory model that can make use of spatiotemporal contextual information from the recent past.

arXiv commentICLR 2021

References in corpus (11)

• Overcoming catastrophic forgetting in neural networks
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Improved Regularization of Convolutional Neural Networks with Cutout
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• FearNet: Brain-Inspired Model for Incremental Learning
• Online Meta-Learning
• Infinite Mixture Prototypes for Few-Shot Learning
• Defining Benchmarks for Continual Few-Shot Learning
• Few-Shot Class-Incremental Learning
• Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning

Cited by in corpus (4)

• Recent Advances of Continual Learning in Computer Vision: An Overview
• Lifelong Adaptive Machine Learning for Sensor-based Human Activity Recognition Using Prototypical Networks
• Variable-Shot Adaptation for Online Meta-Learning
• Expanding continual few-shot learning benchmarks to include recognition of specific instances