Predicting Model Failure using Saliency Maps in Autonomous Driving Systems
arXiv:1905.07679
Abstract
While machine learning systems show high success rate in many complex tasks, research shows they can also fail in very unexpected situations. Rise of machine learning products in safety-critical industries cause an increase in attention in evaluating model robustness and estimating failure probability in machine learning systems. In this work, we propose a design to train a student model -- a failure predictor -- to predict the main model's error for input instances based on their saliency map. We implement and review the preliminary results of our failure predictor model on an autonomous vehicle steering control system as an example of safety-critical applications.
Presented at ICML 2019 Workshop on Uncertainty and Robustness in Deep Learning
References in corpus (12)
- Adam: A Method for Stochastic Optimization
- End to End Learning for Self-Driving Cars
- What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
- Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
- Sanity Checks for Saliency Maps
- Enabling Pedestrian Safety using Computer Vision Techniques: A Case Study of the 2018 Uber Inc. Self-driving Car Crash
- Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Control
- A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learning
- VisualBackProp: efficient visualization of CNNs
- Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration
- Rigorous Agent Evaluation: An Adversarial Approach to Uncover Catastrophic Failures
- Not Using the Car to See the Sidewalk: Quantifying and Controlling the Effects of Context in Classification and Segmentation