Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors

Abstract

A growing body of literature has proposed formal approaches to audit algorithmic systems for biased and harmful behaviors. While formal auditing approaches have been greatly impactful, they often suffer major blindspots, with critical issues surfacing only in the context of everyday use once systems are deployed. Recent years have seen many cases in which everyday users of algorithmic systems detect and raise awareness about harmful behaviors that they encounter in the course of their everyday interactions with these systems. However, to date little academic attention has been granted to these bottom-up, user-driven auditing processes. In this paper, we propose and explore the concept of everyday algorithm auditing, a process in which users detect, understand, and interrogate problematic machine behaviors via their day-to-day interactions with algorithmic systems. We argue that everyday users are powerful in surfacing problematic machine behaviors that may elude detection via more centrally-organized forms of auditing, regardless of users' knowledge about the underlying algorithms. We analyze several real-world cases of everyday algorithm auditing, drawing lessons from these cases for the design of future platforms and tools that facilitate such auditing behaviors. Finally, we discuss work that lies ahead, toward bridging the gaps between formal auditing approaches and the organic auditing behaviors that emerge in everyday use of algorithmic systems.

arXiv commentTo appear in CSCW 2021. The co-first authors and co-senior authors each contributed equally to this work

References in corpus (5)

• Improving fairness in machine learning systems: What do industry practitioners need?
• Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making
• Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media
• Discovering and Validating AI Errors With Crowdsourced Failure Reports
• Understanding the Role of Intermediaries in Online Social E-commerce: An Exploratory Study of Beidian

Cited by in corpus (35)

• Auditing of AI: Legal, Ethical and Technical Approaches
• Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics
• Interactive Model Cards: A Human-Centered Approach to Model Documentation
• Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits
• Supporting Human-AI Collaboration in Auditing LLMs with LLMs
• Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice
• Auditing YouTube's Recommendation Algorithm for Misinformation Filter Bubbles
• Cruising Queer HCI on the DL: A Literature Review of LGBTQ+ People in HCI
• The Algorithmic Imprint
• Sociotechnical Audits: Broadening the Algorithm Auditing Lens to Investigate Targeted Advertising
• A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations
• Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling
• Misfitting With AI: How Blind People Verify and Contest AI Errors
• Youth as Peer Auditors: Engaging Teenagers with Algorithm Auditing of Machine Learning Applications
• Out of Context: Investigating the Bias and Fairness Concerns of "Artificial Intelligence as a Service"
• Participation and Division of Labor in User-Driven Algorithm Audits: How Do Everyday Users Work together to Surface Algorithmic Harms?
• Angler: Helping Machine Translation Practitioners Prioritize Model Improvements
• Wikibench: Community-Driven Data Curation for AI Evaluation on Wikipedia
• Having your Privacy Cake and Eating it Too: Platform-supported Auditing of Social Media Algorithms for Public Interest
• The Fall of an Algorithm: Characterizing the Dynamics Toward Abandonment
• Un-Straightening Generative AI: How Queer Artists Surface and Challenge the Normativity of Generative AI Models
• Youth as Advisors in Participatory Design: Situating Teens' Expertise in Everyday Algorithm Auditing with Teachers and Researchers
• (Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court
• Legacy Procurement Practices Shape How U.S. Cities Govern AI: Understanding Government Employees' Practices, Challenges, and Needs
• Access Denied: Meaningful Data Access for Quantitative Algorithm Audits
• Impact Assessment Card: Communicating Risks and Benefits of AI Uses
• Learning AI Auditing: A Case Study of Teenagers Auditing a Generative AI Model
• End User Authoring of Personalized Content Classifiers: Comparing Example Labeling, Rule Writing, and LLM Prompting
• Clarify: Improving Model Robustness With Natural Language Corrections
• Algorithmic Collective Action with Two Collectives
• Vipera: Towards systematic auditing of generative text-to-image models at scale
• Towards a multi-stakeholder value-based assessment framework for algorithmic systems
• Better Together? The Role of Explanations in Supporting Novices in Individual and Collective Deliberations about AI
• Policy Maps: Tools for Guiding the Unbounded Space of LLM Behaviors
• The Fragility of AI Companionship: Ontological, Structural, and Normative Uncertainty in Human-AI Relationships