Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Flower Hub: A Reproducible Benchmarking Platform for Federated Learning in Simulation and Deployment
Yan Gao, Mohammad Naseri, Javier Fernandez-Marques +19
Federated learning (FL) has emerged as a key approach for training models across decentralized data, yet benchmarking in FL remains difficult to reproduce, compare, and extend. Exi…
cs.LG2025
When Judgment Becomes Noise: How Design Failures in LLM Judge Benchmarks Silently Undermine Validity
Benjamin Feuer, Chiung-Yi Tseng, Astitwa Sarthak Lathe +2
LLM-judged benchmarks are increasingly used to evaluate complex model behaviors, yet their design introduces failure modes absent in conventional ground-truth based benchmarks. We…
cs.LG2024
Style Outweighs Substance: Failure Modes of LLM Judges in Alignment Benchmarking
Benjamin Feuer, Micah Goldblum, Teresa Datta +5
The release of ChatGPT in November 2022 sparked an explosion of interest in post-training and an avalanche of new preference optimization (PO) methods. These methods claim superior…