A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations

Abstract

Responsible design of AI systems is a shared goal across HCI and AI communities. Responsible AI (RAI) tools have been developed to support practitioners to identify, assess, and mitigate ethical issues during AI development. These tools take many forms (e.g., design playbooks, software toolkits, documentation protocols). However, research suggests that use of RAI tools is shaped by organizational contexts, raising questions about how effective such tools are in practice. To better understand how RAI tools are -- and might be -- evaluated, we conducted a qualitative analysis of 37 publications that discuss evaluations of RAI tools. We find that most evaluations focus on usability, while questions of tools' effectiveness in changing AI development are sidelined. While usability evaluations are an important approach to evaluate RAI tools, we draw on evaluation approaches from other fields to highlight developer- and community-level steps to support evaluations of RAI tools' effectiveness in shaping AI development practices and outcomes.

arXiv commentAccepted for publication in Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '24), May 11--16, 2024, Honolulu, HI, USA

References in corpus (29)

• Artificial Intelligence: the global landscape of ethics guidelines
• Model Cards for Model Reporting
• Principles alone cannot guarantee ethical AI
• Improving fairness in machine learning systems: What do industry practitioners need?
• The What-If Tool: Interactive Probing of Machine Learning Models
• Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making
• Evaluation and Measurement of Software Process Improvement -- A Systematic Literature Review
• Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practices
• Measurement and Fairness
• BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation
• Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)
• Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors
• Dislocated Accountabilities in the AI Supply Chain: Modularity and Developers' Notions of Responsibility
• Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics
• Interactive Model Cards: A Human-Centered Approach to Model Documentation
• A Systematic Review and Thematic Analysis of Community-Collaborative Approaches to Computing Research
• Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI
• Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice
• CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation
• Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory
• WEIRD FAccTs: How Western, Educated, Industrialized, Rich, and Democratic is FAccT?
• `It is currently hodgepodge'': Examining AI/ML Practitioners' Challenges during Co-production of Responsible AI Values
• Effects of Sensemaking Translucence on Distributed Collaborative Analysis
• LiFT: A Scalable Framework for Measuring Fairness in ML Applications
• Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloud
• Artificial Concepts of Artificial Intelligence: Institutional Compliance and Resistance in AI Startups
• Exploring AI Futures Through Role Play
• Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools
• On Measures of Biases and Harms in NLP

Cited by in corpus (8)

• Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling
• ASHABot: An LLM-Powered Chatbot to Support the Informational Needs of Community Health Workers
• The Value-Sensitive Conversational Agent Co-Design Framework
• The "Who", "What", and "How" of Responsible AI Governance: A Systematic Review and Meta-Analysis of (Actor, Stage)-Specific Tools
• Impact Assessment Card: Communicating Risks and Benefits of AI Uses
• AI Trust Reshaping Administrative Burdens: Understanding Trust-Burden Dynamics in LLM-Assisted Benefits Systems
• The Turn to Practice in Design Ethics: Characteristics and Future Research Directions for HCI Research
• Compliant But Unsatisfactory: The Gap Between Auditing Standards and Practices for Probabilistic Genotyping Software