8 citations · 20 across the 12 of their papers we have counts for
12 papers
Improving Semantic Uncertainty Quantification in Language Model Question-Answering via Token-Level Temperature Scaling
Tom A. Lamb, Desi R. Ivanova, Philip H. S. Torr +1
Calibration is central to reliable semantic uncertainty quantification, yet prior work has largely focused on discrimination, neglecting calibration. As calibration and discriminat…
JADAI: Jointly Amortizing Adaptive Design and Bayesian Inference
Niels Bracher, Lars Kühmichel, Desi R. Ivanova +3
We consider problems of parameter estimation where design variables can be actively optimized to maximize information gain. To this end, we introduce JADAI, a framework that jointl…
Step-DAD: Semi-Amortized Policy-Based Bayesian Experimental Design
Marcel Hedman, Desi R. Ivanova, Cong Guan +1
We develop a semi-amortized, policy-based, approach to Bayesian experimental design (BED) called Stepwise Deep Adaptive Design (Step-DAD). Like existing, fully amortized, policy-ba…
Extending Epistemic Uncertainty Beyond Parameters Would Assist in Designing Reliable LLMs
T. Duy Nguyen-Hien, Desi R. Ivanova, Yee Whye Teh +1
Although large language models (LLMs) are highly interactive and extendable, current approaches to ensure reliability in deployments remain mostly limited to rejecting outputs with…
Position: Don't Use the CLT in LLM Evals With Fewer Than a Few Hundred Datapoints
Sam Bowyer, Laurence Aitchison, Desi R. Ivanova
Rigorous statistical evaluations of large language models (LLMs), including valid error bars and significance testing, are essential for meaningful and reliable performance assessm…
Is merging worth it? Securely evaluating the information gain for causal dataset acquisition
Jake Fawkes, Lucile Ter-Minassian, Desi Ivanova +2
Merging datasets across institutions is a lengthy and costly procedure, especially when it involves private information. Data hosts may therefore want to prospectively gauge which…
Modern Bayesian Experimental Design
Tom Rainforth, Adam Foster, Desi R Ivanova +1
Bayesian experimental design (BED) provides a powerful and general framework for optimizing the design of experiments. However, its deployment often poses substantial computational…
CO-BED: Information-Theoretic Contextual Optimization via Bayesian Experimental Design
Desi R. Ivanova, Joel Jennings, Tom Rainforth +2
We formalize the problem of contextual optimization through the lens of Bayesian experimental design and propose CO-BED -- a general, model-agnostic framework for designing context…
Differentiable Multi-Target Causal Bayesian Experimental Design
Yashas Annadani, Panagiotis Tigas, Desi R. Ivanova +4
We introduce a gradient-based approach for the problem of Bayesian optimal experimental design to learn causal models in a batch setting -- a critical component for causal discover…
Efficient Real-world Testing of Causal Decision Making via Bayesian Experimental Design for Contextual Optimisation
Desi R. Ivanova, Joel Jennings, Cheng Zhang +1
The real-world testing of decisions made using causal machine learning models is an essential prerequisite for their successful application. We focus on evaluating and improving co…
Implicit Deep Adaptive Design: Policy-Based Experimental Design without Likelihoods
Desi R. Ivanova, Adam Foster, Steven Kleinegesse +2
We introduce implicit Deep Adaptive Design (iDAD), a new method for performing adaptive experiments in real-time with implicit models. iDAD amortizes the cost of Bayesian optimal e…
Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Design
Adam Foster, Desi R. Ivanova, Ilyas Malik +1
We introduce Deep Adaptive Design (DAD), a method for amortizing the cost of adaptive Bayesian experimental design that allows experiments to be run in real-time. Traditional seque…