Localizing Objects with Self-Supervised Transformers and no Labels
arXiv:2109.14279
Abstract
Localizing objects in image collections without supervision can help to avoid expensive annotation campaigns. We propose a simple approach to this problem, that leverages the activation features of a vision transformer pre-trained in a self-supervised manner. Our method, LOST, does not require any external object proposal nor any exploration of the image collection; it operates on a single image. Yet, we outperform state-of-the-art object discovery methods by up to 8 CorLoc points on PASCAL VOC 2012. We also show that training a class-agnostic detector on the discovered objects boosts results by another 7 points. Moreover, we show promising results on the unsupervised object discovery task. The code to reproduce our results can be found at https://github.com/valeoai/LOST.
References in corpus (16)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
- Intriguing Properties of Vision Transformers
- Multi-Object Representation Learning with Iterative Variational Inference
- MONet: Unsupervised Scene Decomposition and Representation
- GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations
- Training Vision Transformers for Image Retrieval
- Self-Supervised Learning with Swin Transformers
- Revisiting the Calibration of Modern Neural Networks
- Toward unsupervised, multi-object discovery in large-scale image collections
- Data Distillation: Towards Omni-Supervised Learning
- Object Discovery From a Single Unlabeled Image by Mining Frequent Itemset With Multi-scale Features
- Unbiased Teacher for Semi-Supervised Object Detection
- Deep Self-Taught Learning for Weakly Supervised Object Localization
- Active Learning for Deep Detection Neural Networks
- PCL: Proposal Cluster Learning for Weakly Supervised Object Detection
Cited by in corpus (8)
- Vision Transformers Need Registers
- A Visual Representation-guided Framework with Global Affinity for Weakly Supervised Salient Object Detection
- Few-Shot Panoptic Segmentation With Foundation Models
- DatUS^2: Data-driven Unsupervised Semantic Segmentation with Pre-trained Self-supervised Vision Transformer
- Disentangling Structure and Appearance in ViT Feature Space
- Utilizing Grounded SAM for self-supervised frugal camouflaged human detection
- SERE: Exploring Feature Self-relation for Self-supervised Transformer
- Three Pillars improving Vision Foundation Model Distillation for Lidar