CoReS: Compatible Representations via Stationarity
arXiv:2111.07632 · doi:10.1109/TPAMI.2023.3259542
Abstract
Compatible features enable the direct comparison of old and new learned features allowing to use them interchangeably over time. In visual search systems, this eliminates the need to extract new features from the gallery-set when the representation model is upgraded with novel data. This has a big value in real applications as re-indexing the gallery-set can be computationally expensive when the gallery-set is large, or even infeasible due to privacy or other concerns of the application. In this paper, we propose CoReS, a new training procedure to learn representations that are \textit{compatible} with those previously learned, grounding on the stationarity of the features as provided by fixed classifiers based on polytopes. With this solution, classes are maximally separated in the representation space and maintain their spatial configuration stationary as new classes are added, so that there is no need to learn any mappings between representations nor to impose pairwise training with the previously learned model. We demonstrate that our training procedure largely outperforms the current state of the art and is particularly effective in the case of multiple upgrades of the training-set, which is the typical case in real applications.
arXiv commentin IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. Code: https://github.com/NiccoBiondi/cores-compatibility
References in corpus (29)
- Adam: A Method for Stochastic Optimization
- FaceNet: A Unified Embedding for Face Recognition and Clustering
- A Simple Framework for Contrastive Learning of Visual Representations
- How transferable are features in deep neural networks?
- Deep Learning Face Representation by Joint Identification-Verification
- Learning Face Representation from Scratch
- Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
- Particular object retrieval with integral max-pooling of CNN activations
- Three scenarios for continual learning
- Prevalence of Neural Collapse during the terminal phase of deep learning training
- Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
- What is being transferred in transfer learning?
- A survey of advances in vision-based vehicle re-identification
- Uniform convergence may be unable to explain generalization in deep learning
- Torchreid: A Library for Deep Learning Person Re-Identification in Pytorch
- Understanding Dimensional Collapse in Contrastive Self-supervised Learning
- Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training
- A Geometric Analysis of Neural Collapse with Unconstrained Features
- Class-incremental Learning with Pre-allocated Fixed Classifiers
- Memory Based Online Learning of Deep Representations from Video Streams
- Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representation
- Regular Polytope Networks
- Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?
- The Met Dataset: Instance-level Recognition for Artworks
- Fix your classifier: the marginal value of training the last weight layer
- Neural Collapse: A Review on Modelling Principles and Generalization
- Backward-Compatible Prediction Updates: A Probabilistic Approach
- Maximally Compact and Separated Features with Regular Polytope Networks
- Unified Representation Learning for Cross Model Compatibility