`It is currently hodgepodge'': Examining AI/ML Practitioners' Challenges during Co-production of Responsible AI Values

Abstract

Recently, the AI/ML research community has indicated an urgent need to establish Responsible AI (RAI) values and practices as part of the AI/ML lifecycle. Several organizations and communities are responding to this call by sharing RAI guidelines. However, there are gaps in awareness, deliberation, and execution of such practices for multi-disciplinary ML practitioners. This work contributes to the discussion by unpacking co-production challenges faced by practitioners as they align their RAI values. We interviewed 23 individuals, across 10 organizations, tasked to ship AI/ML based products while upholding RAI norms and found that both top-down and bottom-up institutional structures create burden for different roles preventing them from upholding RAI values, a challenge that is further exacerbated when executing conflicted values. We share multiple value levers used as strategies by the practitioners to resolve their challenges. We end our paper with recommendations for inclusive and equitable RAI value-practices, creating supportive organizational structures and opportunities to further aid practitioners.

References in corpus (24)

• Artificial Intelligence: the global landscape of ethics guidelines
• Principles alone cannot guarantee ethical AI
• The Ethics of AI Ethics -- An Evaluation of Guidelines
• Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence
• Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions
• What Does Explainable AI Really Mean? A New Conceptualization of Perspectives
• Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practices
• Problem Formulation and Fairness
• "Brilliant AI Doctor" in Rural China: Tensions and Challenges in AI-Powered CDSS Deployment
• The Forgotten Margins of AI Ethics
• How Different Groups Prioritize Ethical Values for Responsible AI
• The Contestation of Tech Ethics: A Sociotechnical Approach to Technology Ethics in Practice
• Envisioning Communities: A Participatory Approach Towards AI for Social Good
• Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems
• Modeling Assumptions Clash with the Real World: Transparency, Equity, and Community Challenges for Student Assignment Algorithms
• Black Feminist Musings on Algorithmic Oppression
• Artificial Intelligence for Social Good: A Survey
• The Key Concepts of Ethics of Artificial Intelligence - A Keyword based Systematic Mapping Study
• The Role of Cooperation in Responsible AI Development
• Algorithmic Fairness in Education
• Stakeholder Participation in AI: Beyond "Add Diverse Stakeholders and Stir"
• Enabling Value Sensitive AI Systems through Participatory Design Fictions
• Building Ethics into Artificial Intelligence
• Towards a multi-stakeholder value-based assessment framework for algorithmic systems

Cited by in corpus (12)

• Farsight: Fostering Responsible AI Awareness During AI Application Prototyping
• A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations
• Sketching AI Concepts with Capabilities and Examples: AI Innovation in the Intensive Care Unit
• Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature
• Designing for Human-Agent Alignment: Understanding what humans want from their agents
• The Value-Sensitive Conversational Agent Co-Design Framework
• The "Who", "What", and "How" of Responsible AI Governance: A Systematic Review and Meta-Analysis of (Actor, Stage)-Specific Tools
• Resolving Ethics Trade-offs in Implementing Responsible AI
• "Come to us first": Centering Community Organizations in Artificial Intelligence for Social Good Partnerships
• Identifying the Barriers to Human-Centered Design in the Workplace: Perspectives from UX Professionals
• AI Trust Reshaping Administrative Burdens: Understanding Trust-Burden Dynamics in LLM-Assisted Benefits Systems
• Talking About the Assumption in the Room