To Trust or Not to Trust a Regressor: Estimating and Explaining Trustworthiness of Regression Predictions

Abstract

In hybrid human-AI systems, users need to decide whether or not to trust an algorithmic prediction while the true error in the prediction is unknown. To accommodate such settings, we introduce RETRO-VIZ, a method for (i) estimating and (ii) explaining trustworthiness of regression predictions. It consists of RETRO, a quantitative estimate of the trustworthiness of a prediction, and VIZ, a visual explanation that helps users identify the reasons for the (lack of) trustworthiness of a prediction. We find that RETRO-scores negatively correlate with prediction error across 117 experimental settings, indicating that RETRO provides a useful measure to distinguish trustworthy predictions from untrustworthy ones. In a user study with 41 participants, we find that VIZ-explanations help users identify whether a prediction is trustworthy or not: on average, 95.1% of participants correctly select the more trustworthy prediction, given a pair of predictions. In addition, an average of 75.6% of participants can accurately describe why a prediction seems to be (not) trustworthy. Finally, we find that the vast majority of users subjectively experience RETRO-VIZ as a useful tool to assess the trustworthiness of algorithmic predictions.

arXiv commentAccepted to ICML 2021 Workshop on Human in the Loop Learning (HILL)

References in corpus (11)

• Towards A Rigorous Science of Interpretable Machine Learning
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• On Calibration of Modern Neural Networks
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• To Trust Or Not To Trust A Classifier
• Metrics for Explainable AI: Challenges and Prospects
• Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions
• Getting a CLUE: A Method for Explaining Uncertainty Estimates
• Investigating Human + Machine Complementarity for Recidivism Predictions
• Harnessing Explanations to Bridge AI and Humans
• Accurate Layerwise Interpretable Competence Estimation

Cited by in corpus (2)

• Uncertainty Estimation and Out-of-Distribution Detection for Counterfactual Explanations: Pitfalls and Solutions
• A Trustworthiness Score to Evaluate DNN Predictions