6 citations · 15 across the 12 of their papers we have counts for
10 papers · 1 filter
LLMs Silently Correct African American English: Auditing and Mitigating Dialect Bias via Activation Steering
Huan Wu, Ali Emami, Muhammad Furquan Hassan +5
African American English (AAE), a rule-governed dialect spoken by over 30 million people, is routinely misinterpreted and "corrected" by large language models (LLMs). Across six in…
Pluralis v0.1: Towards a Multicultural, Multimodal, Multilingual Benchmark for AI Risk and Reliability
Alicia Parrish, Rajat Shinde, Sanket Badhe +57
Current AI safety evaluation and benchmarking frameworks predominantly rely on Western-centric culture-agnostic defaults that mask critical regional laws, socio-linguistic nuances,…
Mitigating Social Biases in Language Models through Unlearning
Omkar Dige, Diljot Singh, Tsz Fung Yau +4
Mitigating bias in language models (LMs) has become a critical problem due to the widespread deployment of LMs. Numerous approaches revolve around data pre-processing and fine-tuni…
Red-Teaming for Inducing Societal Bias in Large Language Models
Chu Fei Luo, Ahmad Ghawanmeh, Bharat Bhimshetty +4
Ensuring the safe deployment of AI systems is critical in industry settings where biased outputs can lead to significant operational, reputational, and regulatory risks. Thorough e…
Template-Based Probes Are Imperfect Lenses for Counterfactual Bias Evaluation in LLMs
Farnaz Kohankhaki, D. B. Emerson, Jacob-Junqi Tian +2
Bias in large language models (LLMs) has many forms, from overt discrimination to implicit stereotypes. Counterfactual bias evaluation is a widely used approach to quantifying bias…
On The Role of Reasoning in the Identification of Subtle Stereotypes in Natural Language
Jacob-Junqi Tian, Omkar Dige, D. B. Emerson +1
Large language models (LLMs) are trained on vast, uncurated datasets that contain various forms of biases and language reinforcing harmful stereotypes that may be subsequently inhe…
Can Instruction Fine-Tuned Language Models Identify Social Bias through Prompting?
Omkar Dige, Jacob-Junqi Tian, David Emerson +1
As the breadth and depth of language model applications continue to expand rapidly, it is increasingly important to build efficient frameworks for measuring and mitigating the lear…
Soft-prompt Tuning for Large Language Models to Evaluate Bias
Jacob-Junqi Tian, David Emerson, Sevil Zanjani Miyandoab +3
Prompting large language models has gained immense popularity in recent years due to the advantage of producing good results even without the need for labelled data. However, this…
Bringing the State-of-the-Art to Customers: A Neural Agent Assistant Framework for Customer Service Support
Stephen Obadinma, Faiza Khan Khattak, Shirley Wang +23
Building Agent Assistants that can help improve customer service support requires inputs from industry users and their customers, as well as knowledge about state-of-the-art Natura…
An Experimental Evaluation of Transformer-based Language Models in the Biomedical Domain
Paul Grouchy, Shobhit Jain, Michael Liu +7
With the growing amount of text in health data, there have been rapid advances in large pre-trained models that can be applied to a wide variety of biomedical tasks with minimal ta…