Multivariate Confidence Calibration for Object Detection

Abstract

Unbiased confidence estimates of neural networks are crucial especially for safety-critical applications. Many methods have been developed to calibrate biased confidence estimates. Though there is a variety of methods for classification, the field of object detection has not been addressed yet. Therefore, we present a novel framework to measure and calibrate biased (or miscalibrated) confidence estimates of object detection methods. The main difference to related work in the field of classifier calibration is that we also use additional information of the regression output of an object detector for calibration. Our approach allows, for the first time, to obtain calibrated confidence estimates with respect to image location and box scale. In addition, we propose a new measure to evaluate miscalibration of object detectors. Finally, we show that our developed methods outperform state-of-the-art calibration models for the task of object detection and provides reliable confidence estimates across different locations and scales.

Accepted on CVPR 2020 Workshop: "2nd Workshop on Safe Artificial Intelligence for Automated Driving (SAIAD)"

References in corpus (8)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• On Calibration of Modern Neural Networks
• Accurate Uncertainties for Deep Learning Using Calibrated Regression
• Uncertainty Estimation in One-Stage Object Detection
• Calibrating Uncertainties in Object Localization Task

Cited by in corpus (15)

• How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• Connecting Low-Loss Subspace for Personalized Federated Learning
• Confidence Calibration for Object Detection and Segmentation
• Fair-Net: A Network Architecture For Reducing Performance Disparity Between Identifiable Sub-Populations
• Improving Video Instance Segmentation by Light-weight Temporal Uncertainty Estimates
• Domain-Specific Pre-training Improves Confidence in Whole Slide Image Classification
• Towards reliable calcification detection: calibration of uncertainty in coronary optical coherence tomography images
• Synergizing between Self-Training and Adversarial Learning for Domain Adaptive Object Detection
• Probabilistic Spatial Transformer Networks
• False Negative Reduction in Video Instance Segmentation using Uncertainty Estimates
• Post-hoc Models for Performance Estimation of Machine Learning Inference
• IKD+: Reliable Low Complexity Deep Models For Retinopathy Classification
• h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objective
• Calibrating the Full Predictive Class Distribution of 3D Object Detectors for Autonomous Driving