Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluation

Kristian Lum, Jacy Reese Anthis, Kevin Robinson +2

Standard benchmarks of bias and fairness in large language models (LLMs) measure the association between the user attributes stated or implied by a prompt and the LLM's short text…