"Cold, Calculated, and Condescending": How AI Identifies and Explains Ableism Compared to Disabled People
arXiv:2410.03448 · doi:10.1145/3715275.3732128
Abstract
People with disabilities (PwD) regularly encounter ableist hate and microaggressions online. These spaces are generally moderated by machine learning models, but little is known about how effectively AI models identify ableist speech and how well their judgments align with PwD. To investigate this, we curated a first-of-its-kind dataset of 200 social media comments targeted towards PwD, and prompted state-of-the art AI models (i.e., Toxicity Classifiers, LLMs) to score toxicity and ableism for each comment, and explain their reasoning. Then, we recruited 190 participants to similarly rate and explain the harm, and evaluate LLM explanations. Our mixed-methods analysis highlighted a major disconnect: AI underestimated toxicity compared to PwD ratings, while its ableism assessments were sporadic and varied. Although LLMs identified some biases, its explanations were flawed--they lacked nuance, made incorrect assumptions, and appeared judgmental instead of educational. Going forward, we discuss challenges and opportunities in designing moderation systems for ableism, and advocate for the involvement of intersectional disabled perspectives in AI.
References in corpus (7)
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT
- An Autoethnographic Case Study of Generative Artificial Intelligence's Utility for Accessibility
- Toward a Perspectivist Turn in Ground Truthing for Predictive Computing
- Better Zero-Shot Reasoning with Role-Play Prompting
- "I Got Flagged for Supposed Bullying, Even Though It Was in Response to Someone Harassing Me About My Disability.": A Study of Blind TikTokers' Content Moderation Experiences
- Mitigating Racial Biases in Toxic Language Detection with an Equity-Based Ensemble Framework