4 citations · 17 across the 14 of their papers we have counts for
16 papers
Distribution Matching Evolutionary Algorithms for Rare Event Sampling
Yonatan Gideoni, Yarin Gal
A novel discovery is one which is both useful and surprising: a generative model's output is a useful discovery if it has a low probability of being generated (it's surprising) and…
Dissecting Training-Free Uncertainty Estimation in Multimodal Large Language Models
Soroush Seifi, Vaggelis Dorovatas, Lin Li +2
Multimodal Large Language Models (MLLMs) are increasingly deployed in safety-critical settings, yet estimating when their answers can be trusted remains underexplored. We systemati…
Building Reliable Long-Form Generation via Hallucination Rejection Sampling
Lin Li, Georgia Channing, Suhaas M Bhat +2
Large language models (LLMs) have achieved remarkable progress in open-ended text generation, yet they remain prone to hallucinating incorrect or unsupported content, which undermi…
Training Transformers for KV Cache Compressibility
Yoav Gelberg, Yam Eitan, Michael Bronstein +2
Long-context language modeling is increasingly constrained by the Key-Value (KV) cache, whose memory and decode-time access costs scale linearly with the prefix length. This bottle…
Muon is Not That Special: Random or Inverted Spectra Work Just as Well
Zakhar Shumaylov, Nathaël Da Costa, Peter Zaika +6
The recent empirical success of the Muon optimizer has renewed interest in non-Euclidean optimization, typically justified by similarities with second-order methods, and linear min…
Selective Safety Steering via Value-Filtered Decoding
Bat-Sheva Einbinder, Hen Davidov, Yee Whye Teh +2
While large language models (LLMs) are trained to align with human values, their generations may still violate safety constraints. A growing line of work addresses this problem by…
Uncertainty Quantification for LLM Function-Calling
Zihuiwen Ye, Lukas Aichberger, Michael Kirchhof +5
Large Language Models (LLMs) are increasingly deployed to autonomously solve real-world tasks. A key ingredient for this is the LLM Function-Calling paradigm, a widely used approac…
Simple Baselines are Competitive with Code Evolution
Yonatan Gideoni, Sebastian Risi, Yarin Gal
Code evolution is a family of techniques that rely on large language models to search through possible computer programs by evolving existing code. While code evolution pipelines h…
MADE: Benchmark Environments for Closed-Loop Materials Discovery
Shreshth A Malik, Tiarnan Doherty, Panagiotis Tigas +4
Existing benchmarks for computational materials discovery primarily evaluate static predictive tasks or isolated computational sub-tasks. While valuable, these evaluations neglect…
Scaling Up Active Testing to Large Language Models
Gabrielle Berrada, Jannik Kossen, Freddie Bickford Smith +3
Active testing enables label-efficient evaluation of predictive models through careful data acquisition, but it can pose a significant computational cost. We identify cost-saving m…
Reducing Large Language Model Safety Risks in Women's Health using Semantic Entropy
Jahan C. Penny-Dimri, Magdalena Bachmann, William R. Cooke +7
Large language models (LLMs) hold substantial promise for clinical decision support. However, their widespread adoption in medicine, particularly in healthcare, is hindered by thei…
Do Multilingual LLMs Think In English?
Lisa Schut, Yarin Gal, Sebastian Farquhar
Large language models (LLMs) have multilingual capabilities and can solve tasks across various languages. However, we show that current LLMs make key decisions in a representation…
Fine-Tuning Large Language Models to Appropriately Abstain with Semantic Entropy
Benedict Aaron Tjandra, Muhammed Razzak, Jannik Kossen +2
Large Language Models (LLMs) are known to hallucinate, whereby they generate plausible but inaccurate text. This phenomenon poses significant risks in critical applications, such a…
Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs
Jannik Kossen, Jiatong Han, Muhammed Razzak +3
We propose semantic entropy probes (SEPs), a cheap and reliable method for uncertainty quantification in Large Language Models (LLMs). Hallucinations, which are plausible-sounding…
Estimating the Hallucination Rate of Generative AI
Andrew Jesson, Nicolas Beltran-Velez, Quentin Chu +5
This paper presents a method for estimating the hallucination rate for in-context learning (ICL) with generative AI. In ICL, a conditional generative model (CGM) is prompted with a…
Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities
Alexander Nikitin, Jannik Kossen, Yarin Gal +1
Uncertainty quantification in Large Language Models (LLMs) is crucial for applications where safety and reliability are important. In particular, uncertainty can be used to improve…