Misfitting With AI: How Blind People Verify and Contest AI Errors

Abstract

Blind people use artificial intelligence-enabled visual assistance technologies (AI VAT) to gain visual access in their everyday lives, but these technologies are embedded with errors that may be difficult to verify non-visually. Previous studies have primarily explored sighted users' understanding of AI output and created vision-dependent explainable AI (XAI) features. We extend this body of literature by conducting an in-depth qualitative study with 26 blind people to understand their verification experiences and preferences. We begin by describing errors blind people encounter, highlighting how AI VAT fails to support complex document layouts, diverse languages, and cultural artifacts. We then illuminate how blind people make sense of AI through experimenting with AI VAT, employing non-visual skills, strategically including sighted people, and cross-referencing with other devices. Participants provided detailed opportunities for designing accessible XAI, such as affordances to support contestation. Informed by disability studies framework of misfitting and fitting, we unpacked harmful assumptions with AI VAT, underscoring the importance of celebrating disabled ways of knowing. Lastly, we offer practical takeaways for Responsible AI practice to push the field of accessible XAI forward.

To Appear in ASSETS 2024

References in corpus (8)

• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making
• On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection
• Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors
• An Autoethnographic Case Study of Generative Artificial Intelligence's Utility for Accessibility
• The Effect of Computer-Generated Descriptions on Photo-Sharing Experiences of People with Visual Impairments
• Humans, AI, and Context: Understanding End-Users' Trust in a Real-World Computer Vision Application
• Contributing to Accessibility Datasets: Reflections on Sharing Study Data by Blind People
• One Explanation is Not Enough: Structured Attention Graphs for Image Classification

Cited by in corpus (2)

• Towards Understanding the Use of MLLM-Enabled Applications for Visual Interpretation by Blind and Low Vision People
• Surfacing Variations to Calibrate Perceived Reliability of MLLM-generated Image Descriptions