5 papers
When Text and Numbers Disagree: Evidence Arbitration in Large Language Models
Mattia Carletti, Edward Phillips, Fredrik K. Gustafsson +5
Large language models (LLMs) are increasingly used in settings where textual summaries, numerical observations, and external tool outputs may provide conflicting evidence. We study…
BAS: A Decision-Theoretic Approach to Evaluating Large Language Model Confidence
Sean Wu, Fredrik K. Gustafsson, Edward Phillips +3
Large language models (LLMs) often produce confident but incorrect answers in settings where abstention would be safer. Standard evaluation protocols, however, require a response a…
Entropy Alone is Insufficient for Safe Selective Prediction in LLMs
Edward Phillips, Fredrik K. Gustafsson, Sean Wu +2
Selective prediction systems can mitigate harms resulting from language model hallucinations by abstaining from answering in high-risk cases. Uncertainty quantification techniques…
Semantic Self-Distillation for Language Model Uncertainty
Edward Phillips, Sean Wu, Fredrik K. Gustafsson +2
Large language models present challenges for principled uncertainty quantification, in part due to their complexity and the diversity of their outputs. Semantic dispersion, or the…
Geometric Uncertainty for Detecting and Correcting Hallucinations in LLMs
Edward Phillips, Sean Wu, Soheila Molaei +3
Large language models are known to hallucinate, generating linguistically plausible but incorrect answers to questions. Uncertainty quantification has been proposed as a strategy t…