4 papers
LLMs Silently Correct African American English: Auditing and Mitigating Dialect Bias via Activation Steering
Huan Wu, Ali Emami, Muhammad Furquan Hassan +5
African American English (AAE), a rule-governed dialect spoken by over 30 million people, is routinely misinterpreted and "corrected" by large language models (LLMs). Across six in…
Pluralis v0.1: Towards a Multicultural, Multimodal, Multilingual Benchmark for AI Risk and Reliability
Alicia Parrish, Rajat Shinde, Sanket Badhe +57
Current AI safety evaluation and benchmarking frameworks predominantly rely on Western-centric culture-agnostic defaults that mask critical regional laws, socio-linguistic nuances,…
Template-Based Probes Are Imperfect Lenses for Counterfactual Bias Evaluation in LLMs
Farnaz Kohankhaki, D. B. Emerson, Jacob-Junqi Tian +2
Bias in large language models (LLMs) has many forms, from overt discrimination to implicit stereotypes. Counterfactual bias evaluation is a widely used approach to quantifying bias…
Red-Teaming for Inducing Societal Bias in Large Language Models
Chu Fei Luo, Ahmad Ghawanmeh, Bharat Bhimshetty +4
Ensuring the safe deployment of AI systems is critical in industry settings where biased outputs can lead to significant operational, reputational, and regulatory risks. Thorough e…