How Much Can We Really Trust You? Towards Simple, Interpretable Trust Quantification Metrics for Deep Neural Networks

Abstract

A critical step to building trustworthy deep neural networks is trust quantification, where we ask the question: How much can we trust a deep neural network? In this study, we take a step towards simple, interpretable metrics for trust quantification by introducing a suite of metrics for assessing the overall trustworthiness of deep neural networks based on their behaviour when answering a set of questions. We conduct a thought experiment and explore two key questions about trust in relation to confidence: 1) How much trust do we have in actors who give wrong answers with great confidence? and 2) How much trust do we have in actors who give right answers hesitantly? Based on insights gained, we introduce the concept of question-answer trust to quantify trustworthiness of an individual answer based on confident behaviour under correct and incorrect answer scenarios, and the concept of trust density to characterize the distribution of overall trust for an individual answer scenario. We further introduce the concept of trust spectrum for representing overall trust with respect to the spectrum of possible answer scenarios across correctly and incorrectly answered questions. Finally, we introduce NetTrustScore, a scalar metric summarizing overall trustworthiness. The suite of metrics aligns with past social psychology studies that study the relationship between trust and confidence. Leveraging these metrics, we quantify the trustworthiness of several well-known deep neural network architectures for image recognition to get a deeper understanding of where trust breaks down. The proposed metrics are by no means perfect, but the hope is to push the conversation towards better metrics to help guide practitioners and regulators in producing, deploying, and certifying deep learning solutions that can be trusted to operate in real-world, mission-critical scenarios.

arXiv comment13 pages

References in corpus (16)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Language Models are Few-Shot Learners
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• On Calibration of Modern Neural Networks
• An Analysis of Deep Neural Network Models for Practical Applications
• Many-body quantum state tomography with neural networks
• AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discovery
• Achieving Human Parity in Conversational Speech Recognition
• Deep Learning and its Application to LHC Physics
• To Trust Or Not To Trust A Classifier
• Do Explanations Reflect Decisions? A Machine-centric Strategy to Quantify the Performance of Explainability Algorithms
• All SMILES Variational Autoencoder
• Safety Verification of Deep Neural Networks
• Vulnerability Under Adversarial Machine Learning: Bias or Variance?

Cited by in corpus (4)

• A deep learning model for burn depth classification using ultrasound imaging
• One-shot skill assessment in high-stakes domains with limited data via meta learning
• Insights into Fairness through Trust: Multi-scale Trust Quantification for Financial Deep Learning
• COVID-Net Clinical ICU: Enhanced Prediction of ICU Admission for COVID-19 Patients via Explainability and Trust Quantification