Distilling Localization for Self-Supervised Representation Learning

Abstract

Recent progress in contrastive learning has revolutionized unsupervised representation learning. Concretely, multiple views (augmentations) from the same image are encouraged to map to the similar embeddings, while views from different images are pulled apart. In this paper, through visualizing and diagnosing classification errors, we observe that current contrastive models are ineffective at localizing the foreground object, limiting their ability to extract discriminative high-level features. This is due to the fact that view generation process considers pixels in an image uniformly. To address this problem, we propose a data-driven approach for learning invariance to backgrounds. It first estimates foreground saliency in images and then creates augmentations by copy-and-pasting the foreground onto a variety of backgrounds. The learning still follows the instance discrimination pretext task, so that the representation is trained to disregard background content and focus on the foreground. We study a variety of saliency estimation methods, and find that most methods lead to improvements for contrastive learning. With this approach (DiLo), significant performance is achieved for self-supervised learning on ImageNet classification, and also for object detection on PASCAL VOC and MSCOCO.

arXiv commentAccepted by AAAI2021

References in corpus (16)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• Improved Regularization of Convolutional Neural Networks with Cutout
• Improved Baselines with Momentum Contrastive Learning
• Momentum Contrast for Unsupervised Visual Representation Learning
• Object Detectors Emerge in Deep Scene CNNs
• Learning Representations by Maximizing Mutual Information Across Views
• Contrastive Multiview Coding
• Deep Clustering for Unsupervised Learning of Visual Features
• Large Scale Adversarial Representation Learning
• Local Aggregation for Unsupervised Learning of Visual Embeddings
• Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspective
• Object Discovery with a Copy-Pasting GAN
• InstaBoost: Boosting Instance Segmentation via Probability Map Guided Copy-Pasting
• Transitive Invariance for Self-supervised Visual Representation Learning

Cited by in corpus (4)

• What makes instance discrimination good for transfer learning?
• Look at What I'm Doing: Self-Supervised Spatial Grounding of Narrations in Instructional Videos
• Instance Localization for Self-supervised Detection Pretraining
• SI-Score: An image dataset for fine-grained analysis of robustness to object location, rotation and size