3D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detection

Abstract

As 3D object detection on point clouds relies on the geometrical relationships between the points, non-standard object shapes can hinder a method's detection capability. However, in safety-critical settings, robustness to out-of-domain and long-tail samples is fundamental to circumvent dangerous issues, such as the misdetection of damaged or rare cars. In this work, we substantially improve the generalization of 3D object detectors to out-of-domain data by deforming point clouds during training. We achieve this with 3D-VField: a novel data augmentation method that plausibly deforms objects via vector fields learned in an adversarial fashion. Our approach constrains 3D points to slide along their sensor view rays while neither adding nor removing any of them. The obtained vectors are transferable, sample-independent and preserve shape and occlusions. Despite training only on a standard dataset, such as KITTI, augmenting with our vector fields significantly improves the generalization to differently shaped objects and scenes. Towards this end, we propose and share CrashD: a synthetic dataset of realistic damaged and rare cars, with a variety of crash scenarios. Extensive experiments on KITTI, Waymo, our CrashD and SUN RGB-D show the generalizability of our techniques to out-of-domain data, different models and sensors, namely LiDAR and ToF cameras, for both indoor and outdoor scenes. Our CrashD dataset is available at https://crashd-cars.github.io.

CVPR 2022. Project page: https://3d-vfield.github.io

References in corpus (4)

• Explaining and Harnessing Adversarial Examples
• Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks
• Description of Corner Cases in Automated Driving: Goals and Challenges
• CertainNet: Sampling-free Uncertainty Estimation for Object Detection

Cited by in corpus (6)

• Segmenting Known Objects and Unseen Unknowns without Prior Knowledge
• 3D Adversarial Augmentations for Robust Out-of-Domain Predictions
• Impact, Attention, Influence: Early Assessment of Autonomous Driving Datasets
• Domain generalization of 3D semantic segmentation in autonomous driving
• DPO: Dual-Perturbation Optimization for Test-time Adaptation in 3D Object Detection
• LiDAR Remote Sensing Meets Weak Supervision: Concepts, Methods, and Perspectives