Large-scale Unsupervised Semantic Segmentation
arXiv:2106.03149 · doi:10.1109/TPAMI.2022.3218275
Abstract
Empowered by large datasets, e.g., ImageNet, unsupervised learning on large-scale data has enabled significant advances for classification tasks. However, whether the large-scale unsupervised semantic segmentation can be achieved remains unknown. There are two major challenges: i) we need a large-scale benchmark for assessing algorithms; ii) we need to develop methods to simultaneously learn category and shape representation in an unsupervised manner. In this work, we propose a new problem of large-scale unsupervised semantic segmentation (LUSS) with a newly created benchmark dataset to help the research progress. Building on the ImageNet dataset, we propose the ImageNet-S dataset with 1.2 million training images and 50k high-quality semantic segmentation annotations for evaluation. Our benchmark has a high data diversity and a clear task objective. We also present a simple yet effective method that works surprisingly well for LUSS. In addition, we benchmark related un/weakly/fully supervised methods accordingly, identifying the challenges and possible directions of LUSS. The benchmark and source code is publicly available at https://github.com/LUSSeg.
arXiv commentBenchmark and Source Code: https://github.com/LUSSeg
References in corpus (9)
- Representation Learning with Contrastive Predictive Coding
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
- Res2Net: A New Multi-scale Backbone Architecture
- Improved Baselines with Momentum Contrastive Learning
- Barlow Twins: Self-Supervised Learning via Redundancy Reduction
- STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation
- Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation
- Improvements to context based self-supervised learning
- Self-labelling via simultaneous clustering and representation learning