4 papers
Binary Agreement for Efficient Unsupervised Confidence Calibration of Reasoning LLMs
Thomas Zollo, Jimmy Wang, Richard Zemel
Reasoning language models often lack the calibrated confidence needed for reliable deployment, particularly when correctness labels are unavailable and sampling budgets are limited…
Adaptive Elicitation of Latent Information Using Natural Language
Jimmy Wang, Thomas Zollo, Richard Zemel +1
Eliciting information to reduce uncertainty about a latent entity is a critical task in many application domains, e.g., assessing individual student learning outcomes, diagnosing u…
AExGym: Benchmarks and Environments for Adaptive Experimentation
Jimmy Wang, Ethan Che, Daniel R. Jiang +1
Innovations across science and industry are evaluated using randomized trials (a.k.a. A/B tests). While simple and robust, such static designs are inefficient or infeasible for tes…
Optimization-Driven Adaptive Experimentation
Ethan Che, Daniel R. Jiang, Hongseok Namkoong +1
Real-world experiments involve batched & delayed feedback, non-stationarity, multiple objectives & constraints, and (often some) personalization. Tailoring adaptive methods to addr…