43 citations · 43 across the 1 of their papers we have counts for
3 papers
cs.CL2025
MICE for CATs: Model-Internal Confidence Estimation for Calibrating Agents with Tools
Nishant Subramani, Jason Eisner, Justin Svegliato +3
Tool-using agents that act in the world need to be both useful and safe. Well-calibrated model confidences can be used to weigh the risk versus reward of potential actions, but pri…
cs.CL2024★ 43 cited
LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Texts
Helia Hashemi, Jason Eisner, Corby Rosset +2
This paper introduces a framework for the automated evaluation of natural language texts. A manually constructed rubric describes how to assess multiple dimensions of interest. To…
cs.CL2024
Let's Think Var-by-Var: Large Language Models Enable Ad Hoc Probabilistic Reasoning
Shepard Xia, Brian Lu, Jason Eisner
A hallmark of intelligence is the ability to flesh out underspecified situations using "common sense." We propose to extract that common sense from large language models (LLMs), in…