1 citations · 3 across the 12 of their papers we have counts for
14 papers
Binary Agreement for Efficient Unsupervised Confidence Calibration of Reasoning LLMs
Thomas Zollo, Jimmy Wang, Richard Zemel
Reasoning language models often lack the calibrated confidence needed for reliable deployment, particularly when correctness labels are unavailable and sampling budgets are limited…
Tell Me What To Learn: Generalizing Neural Memory to be Controllable in Natural Language
Max S. Bennett, Thomas P. Zollo, Richard Zemel
Modern machine learning models are deployed in diverse, non-stationary environments where they must continually adapt to new tasks and evolving knowledge. Continual fine-tuning and…
Whom to Query for What: Adaptive Group Elicitation via Multi-Turn LLM Interactions
Ruomeng Ding, Tianwei Gao, Thomas P. Zollo +3
Eliciting information to reduce uncertainty about latent group-level properties from surveys and other collective assessments requires allocating limited questioning effort under r…
Test-Time Warmup for Multimodal Large Language Models
Nikita Rajaneesh, Thomas Zollo, Richard Zemel
Multimodal Large Language Models (MLLMs) hold great promise for advanced reasoning at the intersection of text and images, yet they have not fully realized this potential. MLLMs ty…
Guiding LLM Decision-Making with Fairness Reward Models
Zara Hall, Melanie Subbiah, Thomas P Zollo +2
Large language models are increasingly used to support high-stakes decisions, potentially influencing who is granted bail or receives a loan. Naive chain-of-thought sampling can im…
QuEst: Enhancing Estimates of Quantile-Based Distributional Measures Using Model Predictions
Zhun Deng, Thomas P Zollo, Benjamin Eyre +3
As machine learning models grow increasingly competent, their predictions can supplement scarce or expensive data in various important domains. In support of this paradigm, algorit…
Confidence Calibration in Vision-Language-Action Models
Thomas P Zollo, Richard Zemel
Trustworthy robot behavior requires not only high levels of task success but also that the robot can reliably quantify how likely it is to succeed. To this end, we present a first-…
Adaptive Elicitation of Latent Information Using Natural Language
Jimmy Wang, Thomas Zollo, Richard Zemel +1
Eliciting information to reduce uncertainty about a latent entity is a critical task in many application domains, e.g., assessing individual student learning outcomes, diagnosing u…
Towards Effective Discrimination Testing for Generative AI
Thomas P. Zollo, Nikita Rajaneesh, Richard Zemel +2
Generative AI (GenAI) models present new challenges in regulating against discriminatory behavior. In this paper, we argue that GenAI fairness research still has not met these chal…
Improving Predictor Reliability with Selective Recalibration
Thomas P. Zollo, Zhun Deng, Jake C. Snell +2
A reliable deep learning system should be able to accurately express its confidence with respect to its predictions, a quality known as calibration. One of the most effective ways…
PersonalLLM: Tailoring LLMs to Individual Preferences
Thomas P. Zollo, Andrew Wei Tung Siah, Naimeng Ye +2
As LLMs become capable of complex tasks, there is growing potential for personalized interactions tailored to the subtle and idiosyncratic preferences of the user. We present a pub…
Prompt Risk Control: A Rigorous Framework for Responsible Deployment of Large Language Models
Thomas P. Zollo, Todd Morrill, Zhun Deng +3
The recent explosion in the capabilities of large language models has led to a wave of interest in how best to prompt a model to perform a given task. While it may be tempting to s…
Distribution-Free Statistical Dispersion Control for Societal Applications
Zhun Deng, Thomas P. Zollo, Jake C. Snell +2
Explicit finite-sample statistical guarantees on model performance are an important ingredient in responsible machine learning. Previous work has focused mainly on bounding either…
Quantile Risk Control: A Flexible Framework for Bounding the Probability of High-Loss Predictions
Jake C. Snell, Thomas P. Zollo, Zhun Deng +2
Rigorous guarantees about the performance of predictive algorithms are necessary in order to ensure their responsible use. Previous work has largely focused on bounding the expecte…