On Cognitive Preferences and the Plausibility of Rule-based Models

Abstract

It is conventional wisdom in machine learning and data mining that logical models such as rule sets are more interpretable than other models, and that among such rule-based models, simpler models are more interpretable than more complex ones. In this position paper, we question this latter assumption by focusing on one particular aspect of interpretability, namely the plausibility of models. Roughly speaking, we equate the plausibility of a model with the likeliness that a user accepts it as an explanation for a prediction. In particular, we argue that, all other things being equal, longer explanations may be more convincing than shorter ones, and that the predominant bias for shorter models, which is typically necessary for learning powerful discriminative models, may not be suitable when it comes to user acceptance of the learned models. To that end, we first recapitulate evidence for and against this postulate, and then report the results of an evaluation in a crowd-sourcing study based on about 3.000 judgments. The results do not reveal a strong preference for simple rules, whereas we can observe a weak preference for longer rules in some domains. We then relate these results to well-known cognitive biases such as the conjunction fallacy, the representative heuristic, or the recogition heuristic, and investigate their relation to rule length and plausibility.

arXiv commentV4: Another rewrite of section on interpretability to clarify focus on plausibility and relation to interpretability, comprehensibility, and justifiability

References in corpus (5)

• Deep Learning in Neural Networks: An Overview
• Distributed Representations of Words and Phrases and their Compositionality
• Learning Functional Causal Models with Generative Neural Networks
• Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016)
• Proceedings of NIPS 2016 Workshop on Interpretable Machine Learning for Complex Systems

Cited by in corpus (7)

• Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions
• A review of possible effects of cognitive biases on the interpretation of rule-based machine learning models
• Stop ordering machine learning algorithms by their explainability! A user-centered investigation of performance and explainability
• Explainable Artificial Intelligence for Process Mining: A General Overview and Application of a Novel Local Explanation Approach for Predictive Process Monitoring
• The Grammar of Interactive Explanatory Model Analysis
• QCBA: Improving Rule Classifiers Learned from Quantitative Data by Recovering Information Lost by Discretisation
• Explaining Predictions by Characteristic Rules