Understanding Frontline Workers' and Unhoused Individuals' Perspectives on AI Used in Homeless Services

Abstract

Recent years have seen growing adoption of AI-based decision-support systems (ADS) in homeless services, yet we know little about stakeholder desires and concerns surrounding their use. In this work, we aim to understand impacted stakeholders' perspectives on a deployed ADS that prioritizes scarce housing resources. We employed AI lifecycle comicboarding, an adapted version of the comicboarding method, to elicit stakeholder feedback and design ideas across various components of an AI system's design. We elicited feedback from county workers who operate the ADS daily, service providers whose work is directly impacted by the ADS, and unhoused individuals in the region. Our participants shared concerns and design suggestions around the AI system's overall objective, specific model design choices, dataset selection, and use in deployment. Our findings demonstrate that stakeholders, even without AI knowledge, can provide specific and critical feedback on an AI system's design and deployment, if empowered to do so.

References in corpus (6)

• Improving fairness in machine learning systems: What do industry practitioners need?
• Algorithms and Decision-Making in the Public Sector
• Imagining new futures beyond predictive systems in child welfare: A qualitative study with impacted stakeholders
• Modeling Assumptions Clash with the Real World: Transparency, Equity, and Community Challenges for Student Assignment Algorithms
• Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systems
• Stakeholder Participation in AI: Beyond "Add Diverse Stakeholders and Stir"

Cited by in corpus (19)

• Multimodal Healthcare AI: Identifying and Designing Clinically Relevant Vision-Language Applications for Radiology
• The Situate AI Guidebook: Co-Designing a Toolkit to Support Multi-Stakeholder Early-stage Deliberations Around Public Sector AI Proposals
• Sketching AI Concepts with Capabilities and Examples: AI Innovation in the Intensive Care Unit
• Are We Asking the Right Questions?: Designing for Community Stakeholders' Interactions with AI in Policing
• Human-centered NLP Fact-checking: Co-Designing with Fact-checkers using Matchmaking for AI
• Wikibench: Community-Driven Data Curation for AI Evaluation on Wikipedia
• "This is not a data problem": Algorithms and Power in Public Higher Education in Canada
• The AI-DEC: A Card-based Design Method for User-centered AI Explanations
• PolicyCraft: Supporting Collaborative and Participatory Policy Design through Case-Grounded Deliberation
• A Human-Centered Review of Algorithms in Homelessness Research
• (Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court
• Legacy Procurement Practices Shape How U.S. Cities Govern AI: Understanding Government Employees' Practices, Challenges, and Needs
• Public Technologies Transforming Work of the Public and the Public Sector
• The Datafication of Care in Public Homelessness Services
• MAP: Multi-user Personalization with Collaborative LLM-powered Agents
• Towards Uncertainty Aware Task Delegation and Human-AI Collaborative Decision-Making
• Understanding How Visually Impaired Players Socialize in Mobile Games
• Deploying a Robust Active Preference Elicitation Algorithm on MTurk: Experiment Design, Interface, and Evaluation for COVID-19 Patient Prioritization
• Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks