A General Framework for Uncertainty Estimation in Deep Learning

Abstract

Neural networks predictions are unreliable when the input sample is out of the training distribution or corrupted by noise. Being able to detect such failures automatically is fundamental to integrate deep learning algorithms into robotics. Current approaches for uncertainty estimation of neural networks require changes to the network and optimization process, typically ignore prior knowledge about the data, and tend to make over-simplifying assumptions which underestimate uncertainty. To address these limitations, we propose a novel framework for uncertainty estimation. Based on Bayesian belief networks and Monte-Carlo sampling, our framework not only fully models the different sources of prediction uncertainty, but also incorporates prior data information, e.g. sensor noise. We show theoretically that this gives us the ability to capture uncertainty better than existing methods. In addition, our framework has several desirable properties: (i) it is agnostic to the network architecture and task; (ii) it does not require changes in the optimization process; (iii) it can be applied to already trained architectures. We thoroughly validate the proposed framework through extensive experiments on both computer vision and control tasks, where we outperform previous methods by up to 23% in accuracy.

arXiv commentAccepted for publication in the Robotics and Automation Letters 2020, and for presentation at the International Conference on Robotics and Automation (ICRA) 2020

References in corpus (5)

• Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
• Differential Flatness of Quadrotor Dynamics Subject to Rotor Drag for Accurate Tracking of High-Speed Trajectories
• A 64mW DNN-based Visual Navigation Engine for Autonomous Nano-Drones
• Driving Policy Transfer via Modularity and Abstraction
• Robust Convolutional Neural Networks under Adversarial Noise

Cited by in corpus (25)

• Machine Learning and Deep Learning -- A review for Ecologists
• Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons
• A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving
• Medical Deep Learning -- A systematic Meta-Review
• The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation
• Deep Learning and Knowledge-Based Methods for Computer Aided Molecular Design -- Toward a Unified Approach: State-of-the-Art and Future Directions
• How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review
• A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applications
• Gaussian process regression + deep neural network autoencoder for probabilistic surrogate modeling in nonlinear mechanics of solids
• An Anomaly Detection Method for Satellites Using Monte Carlo Dropout
• Uncertainty Quantification Metrics for Deep Regression
• Safety-aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Driving
• Uncertainty-Aware Self-Supervised Learning of Spatial Perception Tasks
• Evaluating Object (mis)Detection from a Safety and Reliability Perspective: Discussion and Measures
• DUDES: Deep Uncertainty Distillation using Ensembles for Semantic Segmentation
• UncAD: Towards Safe End-to-end Autonomous Driving via Online Map Uncertainty
• Towards Dependable Autonomous Systems Based on Bayesian Deep Learning Components
• Aleatoric uncertainty for Errors-in-Variables models in deep regression
• An Effective, Robust and Fairness-aware Hate Speech Detection Framework
• Improving Robustness of Deep Neural Networks for Aerial Navigation by Incorporating Input Uncertainty
• Peek into the `White-Box': A Field Study on Bystander Engagement with Urban Robot Uncertainty
• Density Uncertainty Quantification with NeRF-Ensembles: Impact of Data and Scene Constraints
• Robust Neural Regression via Uncertainty Learning
• Uncertainties of Satellite-based Essential Climate Variables from Deep Learning
• PLATE: A perception-latency aware estimator,