A Comparison of Uncertainty Estimation Approaches in Deep Learning Components for Autonomous Vehicle Applications
arXiv:2006.15172
Abstract
A key factor for ensuring safety in Autonomous Vehicles (AVs) is to avoid any abnormal behaviors under undesirable and unpredicted circumstances. As AVs increasingly rely on Deep Neural Networks (DNNs) to perform safety-critical tasks, different methods for uncertainty quantification have recently been proposed to measure the inevitable source of errors in data and models. However, uncertainty quantification in DNNs is still a challenging task. These methods require a higher computational load, a higher memory footprint, and introduce extra latency, which can be prohibitive in safety-critical applications. In this paper, we provide a brief and comparative survey of methods for uncertainty quantification in DNNs along with existing metrics to evaluate uncertainty predictions. We are particularly interested in understanding the advantages and downsides of each method for specific AV tasks and types of uncertainty sources.
arXiv commentAccepted Workshop AISafety 2020 - Workshop in Artificial Intelligence Safety
References in corpus (13)
- A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
- Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
- On a Formal Model of Safe and Scalable Self-driving Cars
- Accurate Uncertainties for Deep Learning Using Calibrated Regression
- Deep Ensembles: A Loss Landscape Perspective
- Bayesian Deep Learning and a Probabilistic Perspective of Generalization
- Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration
- Evaluating Bayesian Deep Learning Methods for Semantic Segmentation
- Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Control
- Practical Solutions for Machine Learning Safety in Autonomous Vehicles
- Can We Trust You? On Calibration of a Probabilistic Object Detector for Autonomous Driving
- Revisiting the Evaluation of Uncertainty Estimation and Its Application to Explore Model Complexity-Uncertainty Trade-Off
- Perceptual Attention-based Predictive Control