Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook

Abstract

Artificial intelligence (AI) presents new challenges for the user experience (UX) of products and services. Recently, practitioner-facing resources and design guidelines have become available to ease some of these challenges. However, little research has investigated if and how these guidelines are used, and how they impact practice. In this paper, we investigated how industry practitioners use the People + AI Guidebook. We conducted interviews with 31 practitioners (i.e., designers, product managers) to understand how they use human-AI guidelines when designing AI-enabled products. Our findings revealed that practitioners use the guidebook not only for addressing AI's design challenges, but also for education, cross-functional communication, and for developing internal resources. We uncovered that practitioners desire more support for early phase ideation and problem formulation to avoid AI product failures. We discuss the implications for future resources aiming to help practitioners in designing AI products.

References in corpus (5)

• Improving fairness in machine learning systems: What do industry practitioners need?
• 'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions
• Problem Formulation and Fairness
• How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study
• A Systematic Review and Thematic Analysis of Community-Collaborative Approaches to Computing Research

Cited by in corpus (21)

• Design Principles for Generative AI Applications
• Multimodal Healthcare AI: Identifying and Designing Clinically Relevant Vision-Language Applications for Radiology
• Farsight: Fostering Responsible AI Awareness During AI Application Prototyping
• The Situate AI Guidebook: Co-Designing a Toolkit to Support Multi-Stakeholder Early-stage Deliberations Around Public Sector AI Proposals
• Sketching AI Concepts with Capabilities and Examples: AI Innovation in the Intensive Care Unit
• Designing for Human-Agent Alignment: Understanding what humans want from their agents
• The Value-Sensitive Conversational Agent Co-Design Framework
• Investigating Why Clinicians Deviate from Standards of Care: Liberating Patients from Mechanical Ventilation in the ICU
• Exploring AI Problem Formulation with Children via Teachable Machines
• Stakeholder Participation for Responsible AI Development: Disconnects Between Guidance and Current Practice
• Interrogating AI: Characterizing Emergent Playful Interactions with ChatGPT
• "Come to us first": Centering Community Organizations in Artificial Intelligence for Social Good Partnerships
• Making the Right Thing: Bridging HCI and Responsible AI in Early-Stage AI Concept Selection
• Supporting Industry Computing Researchers in Assessing, Articulating, and Addressing the Potential Negative Societal Impact of Their Work
• AINeedsPlanner: A Workbook to Support Effective Collaboration Between AI Experts and Clients
• "Why do we do this?": Moral Stress and the Affective Experience of Ethics in Practice
• PADTHAI-MM: Principles-based Approach for Designing Trustworthy, Human-centered AI using MAST Methodology
• Identifying the Barriers to Human-Centered Design in the Workplace: Perspectives from UX Professionals
• Exploring the Innovation Opportunities for Pre-trained Models
• Organization Matters: A Qualitative Study of Organizational Dynamics in Red Teaming Practices for Generative AI
• Talking About the Assumption in the Room