4 papers
Investigating Intersectional Bias in Large Language Models using Confidence Disparities in Coreference Resolution
Falaah Arif Khan, Nivedha Sivakumar, Yinong Oliver Wang +5
Large language models (LLMs) have achieved impressive performance, leading to their widespread adoption as decision-support tools in resource-constrained contexts like hiring and a…
Still More Shades of Null: An Evaluation Suite for Responsible Missing Value Imputation
Falaah Arif Khan, Denys Herasymuk, Nazar Protsiv +1
Data missingness is a practical challenge of sustained interest to the scientific community. In this paper, we present Shades-of-Null, an evaluation suite for responsible missing v…
An Epistemic and Aleatoric Decomposition of Arbitrariness to Constrain the Set of Good Models
Falaah Arif Khan, Denys Herasymuk, Nazar Protsiv +1
Recent research reveals that machine learning (ML) models are highly sensitive to minor changes in their training procedure, such as the inclusion or exclusion of a single data poi…
Is Your Model Fairly Certain? Uncertainty-Aware Fairness Evaluation for LLMs
Yinong Oliver Wang, Nivedha Sivakumar, Falaah Arif Khan +6
The recent rapid adoption of large language models (LLMs) highlights the critical need for benchmarking their fairness. Conventional fairness metrics, which focus on discrete accur…