Discovering and Validating AI Errors With Crowdsourced Failure Reports

Abstract

AI systems can fail to learn important behaviors, leading to real-world issues like safety concerns and biases. Discovering these systematic failures often requires significant developer attention, from hypothesizing potential edge cases to collecting evidence and validating patterns. To scale and streamline this process, we introduce crowdsourced failure reports, end-user descriptions of how or why a model failed, and show how developers can use them to detect AI errors. We also design and implement Deblinder, a visual analytics system for synthesizing failure reports that developers can use to discover and validate systematic failures. In semi-structured interviews and think-aloud studies with 10 AI practitioners, we explore the affordances of the Deblinder system and the applicability of failure reports in real-world settings. Lastly, we show how collecting additional data from the groups identified by developers can improve model performance.

References in corpus (9)

• Improving fairness in machine learning systems: What do industry practitioners need?
• Do ImageNet Classifiers Generalize to ImageNet?
• Data Programming: Creating Large Training Sets, Quickly
• FairSight: Visual Analytics for Fairness in Decision Making
• Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration
• Slice-based Learning: A Programming Model for Residual Learning in Critical Data Slices
• Inspector Gadget: A Data Programming-based Labeling System for Industrial Images
• Interactive Classification for Deep Learning Interpretation
• Designing for Collaborative Sensemaking: Using Expert & Non-Expert Crowd

Cited by in corpus (16)

• Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors
• Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits
• Improving Human-AI Collaboration With Descriptions of AI Behavior
• EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteria
• Supporting Human-AI Collaboration in Auditing LLMs with LLMs
• Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice
• Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning
• Participation and Division of Labor in User-Driven Algorithm Audits: How Do Everyday Users Work together to Surface Algorithmic Harms?
• Angler: Helping Machine Translation Practitioners Prioritize Model Improvements
• Wikibench: Community-Driven Data Curation for AI Evaluation on Wikipedia
• The "Who", "What", and "How" of Responsible AI Governance: A Systematic Review and Meta-Analysis of (Actor, Stage)-Specific Tools
• Where Does My Model Underperform? A Human Evaluation of Slice Discovery Algorithms
• Divisi: Interactive Search and Visualization for Scalable Exploratory Subgroup Analysis
• Optimizing Risk-averse Human-AI Hybrid Teams
• Clarify: Improving Model Robustness With Natural Language Corrections
• Vipera: Towards systematic auditing of generative text-to-image models at scale