Spatially Consistent Representation Learning

Abstract

Self-supervised learning has been widely used to obtain transferrable representations from unlabeled images. Especially, recent contrastive learning methods have shown impressive performances on downstream image classification tasks. While these contrastive methods mainly focus on generating invariant global representations at the image-level under semantic-preserving transformations, they are prone to overlook spatial consistency of local representations and therefore have a limitation in pretraining for localization tasks such as object detection and instance segmentation. Moreover, aggressively cropped views used in existing contrastive methods can minimize representation distances between the semantically different regions of a single image. In this paper, we propose a spatially consistent representation learning algorithm (SCRL) for multi-object and location-specific tasks. In particular, we devise a novel self-supervised objective that tries to produce coherent spatial representations of a randomly cropped local region according to geometric translations and zooming operations. On various downstream localization tasks with benchmark datasets, the proposed SCRL shows significant performance improvements over the image-level supervised pretraining as well as the state-of-the-art self-supervised learning methods. Code is available at https://github.com/kakaobrain/scrl

arXiv commentAccepted by CVPR 2021

References in corpus (13)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• A Simple Framework for Contrastive Learning of Visual Representations
• Bootstrap your own latent: A new approach to self-supervised Learning
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Improved Baselines with Momentum Contrastive Learning
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Deformable DETR: Deformable Transformers for End-to-End Object Detection
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• What Makes for Good Views for Contrastive Learning?
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• Large Scale Adversarial Representation Learning
• LF-Net: Learning Local Features from Images
• Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases

Cited by in corpus (5)

• Sparse R-CNN: End-to-End Object Detection with Learnable Proposals
• Sparse DETR: Efficient End-to-End Object Detection with Learnable Sparsity
• Contrastive Learning with Stronger Augmentations
• Unsupervised Object-Level Representation Learning from Scene Images
• Semi-weakly Supervised Contrastive Representation Learning for Retinal Fundus Images