5 papers
A Representation-Level Assessment of Bias Mitigation in Foundation Models
Svetoslav Nizhnichenkov, Rahul Nair, Elizabeth Daly +1
We investigate how successful bias mitigation reshapes the embedding space of encoder-only and decoder-only foundation models, offering an internal audit of model behaviour through…
Aligning Human and LLM Judgments: Insights from EvalAssist on Task-Specific Evaluations and AI-assisted Assessment Strategy Preferences
Zahra Ashktorab, Michael Desmond, Qian Pan +7
Evaluation of large language model (LLM) outputs requires users to make critical judgments about the best outputs across various configurations. This process is costly and takes ti…
Humble AI in the real-world: the case of algorithmic hiring
Rahul Nair, Inge Vejsbjerg, Elizabeth Daly +2
Humble AI (Knowles et al., 2023) argues for cautiousness in AI development and deployments through scepticism (accounting for limitations of statistical learning), curiosity (accou…
Paying Alignment Tax with Contrastive Learning
Buse Sibel Korkmaz, Rahul Nair, Elizabeth M. Daly +1
Current debiasing approaches often result a degradation in model capabilities such as factual accuracy and knowledge retention. Through systematic evaluation across multiple benchm…
Foundation Models at Work: Fine-Tuning for Fairness in Algorithmic Hiring
Buse Sibel Korkmaz, Rahul Nair, Elizabeth M. Daly +3
Foundation models require fine-tuning to ensure their generative outputs align with intended results for specific tasks. Automating this fine-tuning process is challenging, as it t…