Identifying and Improving Disability Bias in GPT-Based Resume Screening

Abstract

As Generative AI rises in adoption, its use has expanded to include domains such as hiring and recruiting. However, without examining the potential of bias, this may negatively impact marginalized populations, including people with disabilities. To address this important concern, we present a resume audit study, in which we ask ChatGPT (specifically, GPT-4) to rank a resume against the same resume enhanced with an additional leadership award, scholarship, panel presentation, and membership that are disability related. We find that GPT-4 exhibits prejudice towards these enhanced CVs. Further, we show that this prejudice can be quantifiably reduced by training a custom GPTs on principles of DEI and disability justice. Our study also includes a unique qualitative analysis of the types of direct and indirect ableism GPT-4 uses to justify its biased decisions and suggest directions for additional bias mitigation work. Additionally, since these justifications are presumably drawn from training data containing real-world biased statements made by humans, our analysis suggests additional avenues for understanding and addressing human bias.

References in corpus (3)

• An Autoethnographic Case Study of Generative Artificial Intelligence's Utility for Accessibility
• Bias Against 93 Stigmatized Groups in Masked Language Models and Downstream Sentiment Classification Tasks
• AI in the Gray: Exploring Moderation Policies in Dialogic Large Language Models vs. Human Answers in Controversial Topics

Cited by in corpus (6)

• Whither Bias Goes, I Will Go: An Integrative, Systematic Review of Algorithmic Bias Mitigation
• "I Hadn't Thought About That": Creators of Human-like AI Weigh in on Ethics And Neurodivergence
• NeuroBridge: Using Generative AI to Bridge Cross-neurotype Communication Differences through Neurotypical Perspective-taking
• Shiny Stories, Hidden Struggles: Investigating the Representation of Disability Through the Lens of LLMs
• Policy Maps: Tools for Guiding the Unbounded Space of LLM Behaviors
• Identifying Implicit Bias in LLM-based Chat AI Toward People with Intellectual Disabilities