Participation in the age of foundation models

Abstract

Growing interest and investment in the capabilities of foundation models has positioned such systems to impact a wide array of public services. Alongside these opportunities is the risk that these systems reify existing power imbalances and cause disproportionate harm to marginalized communities. Participatory approaches hold promise to instead lend agency and decision-making power to marginalized stakeholders. But existing approaches in participatory AI/ML are typically deeply grounded in context - how do we apply these approaches to foundation models, which are, by design, disconnected from context? Our paper interrogates this question. First, we examine existing attempts at incorporating participation into foundation models. We highlight the tension between participation and scale, demonstrating that it is intractable for impacted communities to meaningfully shape a foundation model that is intended to be universally applicable. In response, we develop a blueprint for participatory foundation models that identifies more local, application-oriented opportunities for meaningful participation. In addition to the "foundation" layer, our framework proposes the "subfloor'' layer, in which stakeholders develop shared technical infrastructure, norms and governance for a grounded domain, and the "surface'' layer, in which affected communities shape the use of a foundation model for a specific downstream task. The intermediate "subfloor'' layer scopes the range of potential harms to consider, and affords communities more concrete avenues for deliberation and intervention. At the same time, it avoids duplicative effort by scaling input across relevant use cases. Through three case studies in clinical care, financial services, and journalism, we illustrate how this multi-layer model can create more meaningful opportunities for participation than solely intervening at the foundation layer.

13 pages, 2 figures. Appeared at FAccT '24

References in corpus (4)

• Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems
• Fine-tuning language models to find agreement among humans with diverse preferences
• A Toolbox for Surfacing Health Equity Harms and Biases in Large Language Models
• Queer In AI: A Case Study in Community-Led Participatory AI

Cited by in corpus (8)

• Beyond Preferences in AI Alignment
• Position is Power: System Prompts as a Mechanism of Bias in Large Language Models (LLMs)
• Secondary Stakeholders in AI: Fighting for, Brokering, and Navigating Agency
• Nteasee: Understanding Needs in AI for Health in Africa -- A Mixed-Methods Study of Expert and General Population Perspectives
• Organization Matters: A Qualitative Study of Organizational Dynamics in Red Teaming Practices for Generative AI
• Policy Maps: Tools for Guiding the Unbounded Space of LLM Behaviors
• AI of the People, by the People, for the People: A Social Choice Approach to Collective Control of Artificial Intelligence
• ONION: A Multi-Layered Framework for Participatory ER Design