Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice

Abstract

Recent years have seen growing interest among both researchers and practitioners in user-engaged approaches to algorithm auditing, which directly engage users in detecting problematic behaviors in algorithmic systems. However, we know little about industry practitioners' current practices and challenges around user-engaged auditing, nor what opportunities exist for them to better leverage such approaches in practice. To investigate, we conducted a series of interviews and iterative co-design activities with practitioners who employ user-engaged auditing approaches in their work. Our findings reveal several challenges practitioners face in appropriately recruiting and incentivizing user auditors, scaffolding user audits, and deriving actionable insights from user-engaged audit reports. Furthermore, practitioners shared organizational obstacles to user-engaged auditing, surfacing a complex relationship between practitioners and user auditors. Based on these findings, we discuss opportunities for future HCI research to help realize the potential (and the mitigate risks) of user-engaged auditing in industry practice.

18 pages. In Proceedings of CHI 2023

References in corpus (5)

• Improving fairness in machine learning systems: What do industry practitioners need?
• Jury Learning: Integrating Dissenting Voices into Machine Learning Models
• Empowering Local Communities Using Artificial Intelligence
• Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits
• Discovering and Validating AI Errors With Crowdsourced Failure Reports

Cited by in corpus (16)

• A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations
• Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling
• Wikibench: Community-Driven Data Curation for AI Evaluation on Wikipedia
• The Value-Sensitive Conversational Agent Co-Design Framework
• From Fitting Participation to Forging Relationships: The Art of Participatory ML
• The "Who", "What", and "How" of Responsible AI Governance: A Systematic Review and Meta-Analysis of (Actor, Stage)-Specific Tools
• (Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court
• Access Denied: Meaningful Data Access for Quantitative Algorithm Audits
• "Come to us first": Centering Community Organizations in Artificial Intelligence for Social Good Partnerships
• Making the Right Thing: Bridging HCI and Responsible AI in Early-Stage AI Concept Selection
• Supporting Industry Computing Researchers in Assessing, Articulating, and Addressing the Potential Negative Societal Impact of Their Work
• Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships
• Clarify: Improving Model Robustness With Natural Language Corrections
• Organization Matters: A Qualitative Study of Organizational Dynamics in Red Teaming Practices for Generative AI
• Under manipulations, are some AI models harder to audit?
• Policy Maps: Tools for Guiding the Unbounded Space of LLM Behaviors