Can We Trust You? On Calibration of a Probabilistic Object Detector for Autonomous Driving

Abstract

Reliable uncertainty estimation is crucial for perception systems in safe autonomous driving. Recently, many methods have been proposed to model uncertainties in deep learning based object detectors. However, the estimated probabilities are often uncalibrated, which may lead to severe problems in safety critical scenarios. In this work, we identify such uncertainty miscalibration problems in a probabilistic LiDAR 3D object detection network, and propose three practical methods to significantly reduce errors in uncertainty calibration. Extensive experiments on several datasets show that our methods produce well-calibrated uncertainties, and generalize well between different datasets.

To appear in IROS 2019

Cited by in corpus (10)

• How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review
• A Review of Testing Object-Based Environment Perception for Safe Automated Driving
• Uncertainty for Identifying Open-Set Errors in Visual Object Detection
• A Comparison of Uncertainty Estimation Approaches in Deep Learning Components for Autonomous Vehicle Applications
• Estimating and Evaluating Regression Predictive Uncertainty in Deep Object Detectors
• Labels Are Not Perfect: Improving Probabilistic Object Detection via Label Uncertainty
• Labels Are Not Perfect: Inferring Spatial Uncertainty in Object Detection
• Credit scoring using neural networks and SURE posterior probability calibration
• Calibrating the Full Predictive Class Distribution of 3D Object Detectors for Autonomous Driving
• $f$-Cal: Calibrated aleatoric uncertainty estimation from neural networks for robot perception