How to Train a (Bad) Algorithmic Caseworker: A Quantitative Deconstruction of Risk Assessments in Child-Welfare
arXiv:2203.05695 · doi:10.1145/3491101.3519771
Abstract
Child welfare (CW) agencies use risk assessment tools as a means to achieve evidence-based, consistent, and unbiased decision-making. These risk assessments act as data collection mechanisms and have further evolved into algorithmic systems in recent years. Moreover, several of these algorithms have reinforced biased theoretical constructs and predictors because of the easy availability of structured assessment data. In this study, we critically examine the Washington Assessment of Risk Model (WARM), a prominent risk assessment tool that has been adopted by over 30 states in the United States and has been repurposed into more complex algorithms. We compared WARM against the narrative coding of casenotes written by caseworkers who used WARM. We found significant discrepancies between the casenotes and WARM data where WARM scores did not not mirror caseworkers' notes about family risk. We provide the SIGCHI community with some initial findings from the quantitative de-construction of a child-welfare algorithm.
References in corpus (4)
- A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
- Modeling Assumptions Clash with the Real World: Transparency, Equity, and Community Challenges for Student Assignment Algorithms
- Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systems
- Unpacking Invisible Work Practices, Constraints, and Latent Power Relationships in Child Welfare through Casenote Analysis
Cited by in corpus (7)
- Imagining new futures beyond predictive systems in child welfare: A qualitative study with impacted stakeholders
- Unpacking Invisible Work Practices, Constraints, and Latent Power Relationships in Child Welfare through Casenote Analysis
- Are We Asking the Right Questions?: Designing for Community Stakeholders' Interactions with AI in Policing
- ARTiST: Automated Text Simplification for Task Guidance in Augmented Reality
- A Human-Centered Review of Algorithms in Homelessness Research
- The Datafication of Care in Public Homelessness Services
- Designing Human-Centered Algorithms for the Public Sector: A Case Study of the U.S. Child-Welfare System