Naive-Student: Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation

Abstract

Supervised learning in large discriminative models is a mainstay for modern computer vision. Such an approach necessitates investing in large-scale human-annotated datasets for achieving state-of-the-art results. In turn, the efficacy of supervised learning may be limited by the size of the human annotated dataset. This limitation is particularly notable for image segmentation tasks, where the expense of human annotation is especially large, yet large amounts of unlabeled data may exist. In this work, we ask if we may leverage semi-supervised learning in unlabeled video sequences and extra images to improve the performance on urban scene segmentation, simultaneously tackling semantic, instance, and panoptic segmentation. The goal of this work is to avoid the construction of sophisticated, learned architectures specific to label propagation (e.g., patch matching and optical flow). Instead, we simply predict pseudo-labels for the unlabeled data and train subsequent models with both human-annotated and pseudo-labeled data. The procedure is iterated for several times. As a result, our Naive-Student model, trained with such simple yet effective iterative semi-supervised learning, attains state-of-the-art results at all three Cityscapes benchmarks, reaching the performance of 67.8% PQ, 42.6% AP, and 85.2% mIOU on the test set. We view this work as a notable step towards building a simple procedure to harness unlabeled video sequences and extra images to surpass state-of-the-art performance on core computer vision tasks.

arXiv commentAccepted to ECCV 2020

References in corpus (17)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Distilling the Knowledge in a Neural Network
• Rethinking Atrous Convolution for Semantic Image Segmentation
• ParseNet: Looking Wider to See Better
• YouTube-8M: A Large-Scale Video Classification Benchmark
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• Learning to Segment Object Candidates
• STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation
• Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
• Scalability in Perception for Autonomous Driving: Waymo Open Dataset
• Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation
• DeeperLab: Single-Shot Image Parser
• Learning to Fuse Things and Stuff
• Understanding Convolution for Semantic Segmentation
• Improving Semantic Segmentation via Self-Training
• PolyTransform: Deep Polygon Transformer for Instance Segmentation

Cited by in corpus (7)

• Rethinking Pre-training and Self-training
• Robust Mutual Learning for Semi-supervised Semantic Segmentation
• EC-GAN: Low-Sample Classification using Semi-Supervised Algorithms and GANs
• Pseudo-labeling for Scalable 3D Object Detection
• Semi-supervised Domain Adaptation for Semantic Segmentation
• AinnoSeg: Panoramic Segmentation with High Perfomance
• Semi-supervised Cardiac Image Segmentation via Label Propagation and Style Transfer