Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty

Abstract

Algorithmic transparency entails exposing system properties to various stakeholders for purposes that include understanding, improving, and contesting predictions. Until now, most research into algorithmic transparency has predominantly focused on explainability. Explainability attempts to provide reasons for a machine learning model's behavior to stakeholders. However, understanding a model's specific behavior alone might not be enough for stakeholders to gauge whether the model is wrong or lacks sufficient knowledge to solve the task at hand. In this paper, we argue for considering a complementary form of transparency by estimating and communicating the uncertainty associated with model predictions. First, we discuss methods for assessing uncertainty. Then, we characterize how uncertainty can be used to mitigate model unfairness, augment decision-making, and build trustworthy systems. Finally, we outline methods for displaying uncertainty to stakeholders and recommend how to collect information required for incorporating uncertainty into existing ML pipelines. This work constitutes an interdisciplinary review drawn from literature spanning machine learning, visualization/HCI, design, decision-making, and fairness. We aim to encourage researchers and practitioners to measure, communicate, and use uncertainty as a form of transparency.

arXiv commentAAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES) 2021

References in corpus (49)

• Towards A Rigorous Science of Interpretable Machine Learning
• Axiomatic Attribution for Deep Networks
• Equality of Opportunity in Supervised Learning
• On Calibration of Modern Neural Networks
• Understanding Black-box Predictions via Influence Functions
• Weight Uncertainty in Neural Networks
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
• Bayesian Active Learning for Classification and Preference Learning
• Stochastic Gradient Hamiltonian Monte Carlo
• Variational Dropout and the Local Reparameterization Trick
• Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
• Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
• BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning
• On Fairness and Calibration
• Uncertainty Estimation Using a Single Deep Deterministic Neural Network
• Fairness Without Demographics in Repeated Loss Minimization
• Bayesian Deep Learning and a Probabilistic Perspective of Generalization
• A Simple Baseline for Bayesian Uncertainty in Deep Learning
• Bayesian Uncertainty Estimation for Batch Normalized Deep Networks
• Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration
• Measuring Calibration in Deep Learning
• Evaluating Bayesian Deep Learning Methods for Semantic Segmentation
• Identifying and Correcting Label Bias in Machine Learning
• Functional Variational Bayesian Neural Networks
• Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning
• Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness
• A Reductions Approach to Fair Classification
• How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation
• Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning
• Single-Model Uncertainties for Deep Learning
• SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates
• Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer
• On the Expressiveness of Approximate Inference in Bayesian Neural Networks
• Robust Optimization for Fairness with Noisy Protected Groups
• Machine Learning Explainability for External Stakeholders
• Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learning
• Probabilistic Modeling of Deep Features for Out-of-Distribution and Adversarial Detection
• Assessing Algorithmic Fairness with Unobserved Protected Class Using Data Combination
• Well-calibrated Model Uncertainty with Temperature Scaling for Dropout Variational Inference
• Fairness Evaluation in Presence of Biased Noisy Labels
• ABOUT ML: Annotation and Benchmarking on Understanding and Transparency of Machine Learning Lifecycles
• Successor Uncertainties: Exploration and Uncertainty in Temporal Difference Learning
• Discriminative Jackknife: Quantifying Uncertainty in Deep Learning via Higher-Order Influence Functions
• With Malice Towards None: Assessing Uncertainty via Equalized Coverage
• Entropy and Uncertainty Analysis in Financial Markets
• A Multistakeholder Approach Towards Evaluating AI Transparency Mechanisms
• δ-CLUE: Diverse Sets of Explanations for Uncertainty Estimates

Cited by in corpus (8)

• Getting a CLUE: A Method for Explaining Uncertainty Estimates
• Socially Responsible AI Algorithms: Issues, Purposes, and Challenges
• Question-Driven Design Process for Explainable AI User Experiences
• Can We Leverage Predictive Uncertainty to Detect Dataset Shift and Adversarial Examples in Android Malware Detection?
• Using AntiPatterns to avoid MLOps Mistakes
• Uncertainty Estimation and Out-of-Distribution Detection for Counterfactual Explanations: Pitfalls and Solutions
• Bayesian Deep Learning via Subnetwork Inference
• Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detection