Pseudo-labeling for Scalable 3D Object Detection
arXiv:2103.02093
Abstract
To safely deploy autonomous vehicles, onboard perception systems must work reliably at high accuracy across a diverse set of environments and geographies. One of the most common techniques to improve the efficacy of such systems in new domains involves collecting large labeled datasets, but such datasets can be extremely costly to obtain, especially if each new deployment geography requires additional data with expensive 3D bounding box annotations. We demonstrate that pseudo-labeling for 3D object detection is an effective way to exploit less expensive and more widely available unlabeled data, and can lead to performance gains across various architectures, data augmentation strategies, and sizes of the labeled dataset. Overall, we show that better teacher models lead to better student models, and that we can distill expensive teachers into efficient, simple students. Specifically, we demonstrate that pseudo-label-trained student models can outperform supervised models trained on 3-10 times the amount of labeled examples. Using PointPillars [24], a two-year-old architecture, as our student model, we are able to achieve state of the art accuracy simply by leveraging large quantities of pseudo-labeled data. Lastly, we show that these student models generalize better than supervised models to a new domain in which we only have unlabeled data, making pseudo-label training an effective form of unsupervised domain adaptation.
References in corpus (23)
- Adam: A Method for Stochastic Optimization
- Distilling the Knowledge in a Neural Network
- FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
- Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
- Temporal Ensembling for Semi-Supervised Learning
- Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
- Rethinking Pre-training and Self-training
- Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
- Billion-scale semi-supervised learning for image classification
- A Simple Semi-Supervised Learning Framework for Object Detection
- HDNET: Exploiting HD Maps for 3D Object Detection
- Improved Noisy Student Training for Automatic Speech Recognition
- Scalability in Perception for Autonomous Driving: Waymo Open Dataset
- Self-Training for End-to-End Speech Recognition
- One Thousand and One Hours: Self-driving Motion Prediction Dataset
- ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
- StarNet: Targeted Computation for Object Detection in Point Clouds
- AFDet: Anchor Free One Stage 3D Object Detection
- Auto4D: Learning to Label 4D Objects from Sequential Point Clouds
- 1st Place Solution for Waymo Open Dataset Challenge -- 3D Detection and Domain Adaptation
- Naive-Student: Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation
- Improving 3D Object Detection through Progressive Population Based Augmentation
- Multi-Frame to Single-Frame: Knowledge Distillation for 3D Object Detection