4 citations · 4 across the 3 of their papers we have counts for
3 papers
Quantifying the Effect of Test Set Contamination on Generative Evaluations
Rylan Schaeffer, Joshua Kazdan, Baber Abbasi +8
As frontier AI systems are pretrained on web-scale data, test set contamination has become a critical concern for accurately assessing their capabilities. While research has thorou…
Measurement to Meaning: A Validity-Centered Framework for AI Evaluation
Olawale Salaudeen, Anka Reuel, Ahmed Ahmed +6
While the capabilities and utility of AI systems have advanced, rigorous norms for evaluating these systems have lagged. Grand claims, such as models achieving general reasoning ca…
Scalable Ensembling For Mitigating Reward Overoptimisation
Ahmed M. Ahmed, Rafael Rafailov, Stepan Sharkov +2
Reinforcement Learning from Human Feedback (RLHF) has enabled significant advancements within language modeling for powerful, instruction-following models. However, the alignment o…