CL2R: Compatible Lifelong Learning Representations

Abstract

In this paper, we propose a method to partially mimic natural intelligence for the problem of lifelong learning representations that are compatible. We take the perspective of a learning agent that is interested in recognizing object instances in an open dynamic universe in a way in which any update to its internal feature representation does not render the features in the gallery unusable for visual search. We refer to this learning problem as Compatible Lifelong Learning Representations (CL2R) as it considers compatible representation learning within the lifelong learning paradigm. We identify stationarity as the property that the feature representation is required to hold to achieve compatibility and propose a novel training procedure that encourages local and global stationarity on the learned representation. Due to stationarity, the statistical properties of the learned features do not change over time, making them interoperable with previously learned features. Extensive experiments on standard benchmark datasets show that our CL2R training procedure outperforms alternative baselines and state-of-the-art methods. We also provide novel metrics to specifically evaluate compatible representation learning under catastrophic forgetting in various sequential learning tasks. Code at https://github.com/NiccoBiondi/CompatibleLifelongRepresentation.

arXiv commentPublished on ACM TOMM 2022

References in corpus (22)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Distilling the Knowledge in a Neural Network
• FaceNet: A Unified Embedding for Face Recognition and Clustering
• FitNets: Hints for Thin Deep Nets
• Deep Learning Face Representation by Joint Identification-Verification
• A continual learning survey: Defying forgetting in classification tasks
• Learning Face Representation from Scratch
• Person Re-identification: Past, Present and Future
• Particular object retrieval with integral max-pooling of CNN activations
• A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks
• Deep Learning for Person Re-identification: A Survey and Outlook
• Don't forget, there is more than forgetting: new metrics for Continual Learning
• Memory Based Online Learning of Deep Representations from Video Streams
• Regular Polytope Networks
• FOSTER: Feature Boosting and Compression for Class-Incremental Learning
• A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges
• Continuum: Simple Management of Complex Continual Learning Scenarios
• On the Exploration of Incremental Learning for Fine-grained Image Retrieval
• Fix Your Features: Stationary and Maximally Discriminative Embeddings using Regular Polytope (Fixed Classifier) Networks
• Contrastive Supervised Distillation for Continual Representation Learning
• Maximally Compact and Separated Features with Regular Polytope Networks
• CoReS: Compatible Representations via Stationarity