A review of possible effects of cognitive biases on the interpretation of rule-based machine learning models

Abstract

While the interpretability of machine learning models is often equated with their mere syntactic comprehensibility, we think that interpretability goes beyond that, and that human interpretability should also be investigated from the point of view of cognitive science. The goal of this paper is to discuss to what extent cognitive biases may affect human understanding of interpretable machine learning models, in particular of logical rules discovered from data. Twenty cognitive biases are covered, as are possible debiasing techniques that can be adopted by designers of machine learning algorithms and software. Our review transfers results obtained in cognitive psychology to the domain of machine learning, aiming to bridge the current gap between these two areas. It needs to be followed by empirical studies specifically focused on the machine learning domain.

References in corpus (5)

• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
• The Pragmatic Turn in Explainable Artificial Intelligence (XAI)
• An Evaluation of the Human-Interpretability of Explanation
• How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation
• On Cognitive Preferences and the Plausibility of Rule-based Models

Cited by in corpus (8)

• Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations
• Unpacking Human-AI Interaction in Safety-Critical Industries: A Systematic Literature Review
• Why we need biased AI -- How including cognitive and ethical machine biases can enhance AI systems
• Interpretable Learned Emergent Communication for Human-Agent Teams
• Explaining Predictions by Characteristic Rules
• It's only fair when I think it's fair: How Gender Bias Alignment Undermines Distributive Fairness in Human-AI Collaboration
• Towards Enterprise-Ready AI Deployments Minimizing the Risk of Consuming AI Models in Business Applications
• Deep Active Re-Labeling: Toward Noise-Resilient Annotation Efficiency