MS3D++: Ensemble of Experts for Multi-Source Unsupervised Domain Adaptation in 3D Object Detection

Abstract

Deploying 3D detectors in unfamiliar domains has been demonstrated to result in a significant 70-90% drop in detection rate due to variations in lidar, geography, or weather from their training dataset. This domain gap leads to missing detections for densely observed objects, misaligned confidence scores, and increased high-confidence false positives, rendering the detector highly unreliable. To address this, we introduce MS3D++, a self-training framework for multi-source unsupervised domain adaptation in 3D object detection. MS3D++ generates high-quality pseudo-labels, allowing 3D detectors to achieve high performance on a range of lidar types, regardless of their density. Our approach effectively fuses predictions of an ensemble of multi-frame pre-trained detectors from different source domains to improve domain generalization. We subsequently refine predictions temporally to ensure temporal consistency in box localization and object classification. Furthermore, we present an in-depth study into the performance and idiosyncrasies of various 3D detector components in a cross-domain context, providing valuable insights for improved cross-domain detector ensembling. Experimental results on Waymo, nuScenes and Lyft demonstrate that detectors trained with MS3D++ pseudo-labels achieve state-of-the-art performance, comparable to training with human-annotated labels in Bird's Eye View (BEV) evaluation for both low and high density lidar. Code is available at https://github.com/darrenjkt/MS3D

References in corpus (11)

• Point-Voxel CNN for Efficient 3D Deep Learning
• One Thousand and One Hours: Self-driving Motion Prediction Dataset
• 3DSSD: Point-based 3D Single Stage Object Detector
• See Eye to Eye: A Lidar-Agnostic 3D Detection Framework for Unsupervised Multi-Target Domain Adaptation
• Zenseact Open Dataset: A large-scale and diverse multimodal dataset for autonomous driving
• SF-UDA${}^{3D}$: Source-Free Unsupervised Domain Adaptation for LiDAR-Based 3D Object Detection
• ST3D++: Denoised Self-training for Unsupervised Domain Adaptation on 3D Object Detection
• M3DeTR: Multi-representation, Multi-scale, Mutual-relation 3D Object Detection with Transformers
• Once Detected, Never Lost: Surpassing Human Performance in Offline LiDAR based 3D Object Detection
• Revisiting Domain-Adaptive 3D Object Detection by Reliable, Diverse and Class-balanced Pseudo-Labeling
• DetZero: Rethinking Offboard 3D Object Detection with Long-term Sequential Point Clouds