164 citations · 288 across the 13 of their papers we have counts for
3 papers · 1 filter
Predicting LLM Safety Before Release by Simulating Deployment
Marcus Williams, Hannah Sheahan, Cameron Raymond +8
Pre-deployment safety evaluations aim to inform the downstream risks of releasing a new AI model. Yet most evaluations provide limited evidence about how often undesired model beha…
Whose Ground Truth? Accounting for Individual and Collective Identities Underlying Dataset Annotation
Remi Denton, Mark Díaz, Ian Kivlichan +2
Human annotations play a crucial role in machine learning (ML) research and development. However, the ethical considerations around the processes and decisions that go into buildin…
Measuring and Improving Model-Moderator Collaboration using Uncertainty Estimation
Ian D. Kivlichan, Zi Lin, Jeremiah Liu +1
Content moderation is often performed by a collaboration between humans and machine learning models. However, it is not well understood how to design the collaborative process so a…