3 papers
cs.CL2025
On the Entropy Calibration of Language Models
Steven Cao, Gregory Valiant, Percy Liang
We study the problem of entropy calibration, which asks whether a language model's entropy over generations matches its log loss on human text. Past work found that models are misc…
cs.CL2025
Language Models May Verbatim Complete Text They Were Not Explicitly Trained On
Ken Ziyu Liu, Christopher A. Choquette-Choo, Matthew Jagielski +4
An important question today is whether a given text was used to train a large language model (LLM). A \emph{completion} test is often employed: check if the LLM completes a suffici…
cs.CL2025
Reliable and Efficient Amortized Model-based Evaluation
Sang Truong, Yuheng Tu, Percy Liang +2
Comprehensive evaluations of language models (LM) during both development and deployment phases are necessary because these models possess numerous capabilities (e.g., mathematical…