21 citations · 22 across the 5 of their papers we have counts for
5 papers
Flower Hub: A Reproducible Benchmarking Platform for Federated Learning in Simulation and Deployment
Yan Gao, Mohammad Naseri, Javier Fernandez-Marques +19
Federated learning (FL) has emerged as a key approach for training models across decentralized data, yet benchmarking in FL remains difficult to reproduce, compare, and extend. Exi…
When Judgment Becomes Noise: How Design Failures in LLM Judge Benchmarks Silently Undermine Validity
Benjamin Feuer, Chiung-Yi Tseng, Astitwa Sarthak Lathe +2
LLM-judged benchmarks are increasingly used to evaluate complex model behaviors, yet their design introduces failure modes absent in conventional ground-truth based benchmarks. We…
Style Outweighs Substance: Failure Modes of LLM Judges in Alignment Benchmarking
Benjamin Feuer, Micah Goldblum, Teresa Datta +5
The release of ChatGPT in November 2022 sparked an explosion of interest in post-training and an avalanche of new preference optimization (PO) methods. These methods claim superior…
Strategies for Increasing Corporate Responsible AI Prioritization
Angelina Wang, Teresa Datta, John P. Dickerson
Responsible artificial intelligence (RAI) is increasingly recognized as a critical concern. However, the level of corporate RAI prioritization has not kept pace. In this work, we c…
Large language models that replace human participants can harmfully misportray and flatten identity groups
Angelina Wang, Jamie Morgenstern, John P. Dickerson
Large language models (LLMs) are increasing in capability and popularity, propelling their application in new domains -- including as replacements for human participants in computa…