Mean Accuracy Is the Wrong Estimand for a Release Decision

Jaewon Choi, Jihye Kang, Namhyuk Ahn

A release decision about a fine-tuned model is an operational commitment about the \emph{next} rerun of its training recipe. Yet standard benchmark reporting gives only the mean ac…