8 citations · 20 across the 12 of their papers we have counts for
4 papers · 1 filter
JADAI: Jointly Amortizing Adaptive Design and Bayesian Inference
Niels Bracher, Lars Kühmichel, Desi R. Ivanova +3
We consider problems of parameter estimation where design variables can be actively optimized to maximize information gain. To this end, we introduce JADAI, a framework that jointl…
Step-DAD: Semi-Amortized Policy-Based Bayesian Experimental Design
Marcel Hedman, Desi R. Ivanova, Cong Guan +1
We develop a semi-amortized, policy-based, approach to Bayesian experimental design (BED) called Stepwise Deep Adaptive Design (Step-DAD). Like existing, fully amortized, policy-ba…
Extending Epistemic Uncertainty Beyond Parameters Would Assist in Designing Reliable LLMs
T. Duy Nguyen-Hien, Desi R. Ivanova, Yee Whye Teh +1
Although large language models (LLMs) are highly interactive and extendable, current approaches to ensure reliability in deployments remain mostly limited to rejecting outputs with…
Position: Don't Use the CLT in LLM Evals With Fewer Than a Few Hundred Datapoints
Sam Bowyer, Laurence Aitchison, Desi R. Ivanova
Rigorous statistical evaluations of large language models (LLMs), including valid error bars and significance testing, are essential for meaningful and reliable performance assessm…