Towards Fine-grained Visual Representations by Combining Contrastive Learning with Image Reconstruction and Attention-weighted Pooling
arXiv:2104.04323
Abstract
This paper presents Contrastive Reconstruction, ConRec - a self-supervised learning algorithm that obtains image representations by jointly optimizing a contrastive and a self-reconstruction loss. We showcase that state-of-the-art contrastive learning methods (e.g. SimCLR) have shortcomings to capture fine-grained visual features in their representations. ConRec extends the SimCLR framework by adding (1) a self-reconstruction task and (2) an attention mechanism within the contrastive learning task. This is accomplished by applying a simple encoder-decoder architecture with two heads. We show that both extensions contribute towards an improved vector representation for images with fine-grained visual features. Combining those concepts, ConRec outperforms SimCLR and SimCLR with Attention-Pooling on fine-grained classification datasets.
ICML 2021 Workshop: Self-Supervised Learning for Reasoning and Perception
References in corpus (14)
- A Simple Framework for Contrastive Learning of Visual Representations
- Improved Baselines with Momentum Contrastive Learning
- Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
- Big Self-Supervised Models are Strong Semi-Supervised Learners
- The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification
- Unsupervised Visual Representation Learning by Context Prediction
- 3D Self-Supervised Methods for Medical Imaging
- Learning and Evaluating Representations for Deep One-class Classification
- Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?
- Deep Learning for Fine-Grained Image Analysis: A Survey
- Self-Supervision Closes the Gap Between Weak and Strong Supervision in Histology
- Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysis
- Cross-Domain Adaptation for Animal Pose Estimation
- Fine-Grain Few-Shot Vision via Domain Knowledge as Hyperspherical Priors