"Trust Junk" Leads to Unjustified Support for Highly Discriminatory Predictive Models
arXiv:2607.14152
The paper shows that adding accurate but irrelevant data to model explanations can cause people to over‑trust and positively view discriminatory predictive models, highlighting risks in XAI visualizations.
Abstract
The persuasive power of data visualizations can go awry: for instance, in an explainable AI (XAI) context, visualizations can produce over-trust of predictive models. In this paper, we use a crowdsourced study to show that providing accurate (but superfluous or irrelevant) data in a model explanation can, in fact, result in unjustified trust and other positive beliefs about a model, even when the model is patently discriminatory and unfair. Our results suggest that XAI designers and developers need to consider the implicit or explicit rhetorics of their work, and beware of the potential of visualizations to imbue models with unearned trust.