3DIoUMatch: Leveraging IoU Prediction for Semi-Supervised 3D Object Detection
arXiv:2012.04355
Abstract
3D object detection is an important yet demanding task that heavily relies on difficult to obtain 3D annotations. To reduce the required amount of supervision, we propose 3DIoUMatch, a novel semi-supervised method for 3D object detection applicable to both indoor and outdoor scenes. We leverage a teacher-student mutual learning framework to propagate information from the labeled to the unlabeled train set in the form of pseudo-labels. However, due to the high task complexity, we observe that the pseudo-labels suffer from significant noise and are thus not directly usable. To that end, we introduce a confidence-based filtering mechanism, inspired by FixMatch. We set confidence thresholds based upon the predicted objectness and class probability to filter low-quality pseudo-labels. While effective, we observe that these two measures do not sufficiently capture localization quality. We therefore propose to use the estimated 3D IoU as a localization metric and set category-aware self-adjusted thresholds to filter poorly localized proposals. We adopt VoteNet as our backbone detector on indoor datasets while we use PV-RCNN on the autonomous driving dataset, KITTI. Our method consistently improves state-of-the-art methods on both ScanNet and SUN-RGBD benchmarks by significant margins under all label ratios (including fully labeled setting). For example, when training using only 10\% labeled data on ScanNet, 3DIoUMatch achieves 7.7% absolute improvement on [email protected] and 8.5% absolute improvement on [email protected] upon the prior art. On KITTI, we are the first to demonstrate semi-supervised 3D object detection and our method surpasses a fully supervised baseline from 1.8% to 7.6% under different label ratios and categories.
arXiv commentCVPR 2021
References in corpus (11)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Improved Regularization of Convolutional Neural Networks with Cutout
- FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
- Unsupervised Data Augmentation for Consistency Training
- Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
- Temporal Ensembling for Semi-Supervised Learning
- ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
- A Simple Semi-Supervised Learning Framework for Object Detection
- LSDA: Large Scale Detection Through Adaptation
- ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
- IoU-uniform R-CNN: Breaking Through the Limitations of RPN