160 citations · 314 across the 32 of their papers we have counts for
4 papers · 1 filter
Welfare, Improvability, and Variance: A Principal-Agent Approach to Optimal Benchmark Item Aggregation
Andreas Haupt, Justin Hartenstein, Anka Reuel +2
AI benchmarks have well-documented limitations, with prior work examining contamination, saturation, and construct underspecification. Aggregation has received far less attention:…
Humanity's Last Exam
Long Phan, Alice Gatti, Ziwen Han +1168
Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achi…
Fairness in Reinforcement Learning: A Survey
Anka Reuel, Devin Ma
While our understanding of fairness in machine learning has significantly progressed, our understanding of fairness in reinforcement learning (RL) remains nascent. Most of the atte…
Analyzing And Editing Inner Mechanisms Of Backdoored Language Models
Max Lamparth, Anka Reuel
Poisoning of data sets is a potential security threat to large language models that can lead to backdoored models. A description of the internal mechanisms of backdoored language m…