2 papers
cs.CL2024
LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Texts
Helia Hashemi, Jason Eisner, Corby Rosset +2
This paper introduces a framework for the automated evaluation of natural language texts. A manually constructed rubric describes how to assess multiple dimensions of interest. To…
cs.CL2024
Do Androids Know They're Only Dreaming of Electric Sheep?
Sky CH-Wang, Benjamin Van Durme, Jason Eisner +1
We design probes trained on the internal representations of a transformer language model to predict its hallucinatory behavior on three grounded generation tasks. To train the prob…