7 papers
Beyond Single-Turn Confidence: Trajectory-Adapted Uncertainty Quantification for LLM Agents
Dylan Bouchard, Mohit Singh Chauhan
Uncertainty quantification (UQ) methods for language models are typically evaluated on single-turn outputs, where uncertainty is attached to one generated answer. For LLM agents, h…
Fine-Grained Uncertainty Quantification for Long-Form Language Model Outputs: A Comparative Study
Dylan Bouchard, Mohit Singh Chauhan, Viren Bajaj +1
Uncertainty quantification has emerged as an effective approach to closed-book hallucination detection for LLMs, but existing methods are largely designed for short-form outputs an…
DECK: A Consistency x Confidence Taxonomy of LLM Hallucinations
Mohit Singh Chauhan
Existing hallucination taxonomies classify LLM errors by what is wrong with the output -- memorised misconceptions, reasoning failures, fluent fabrications. These taxonomies are us…
Functional Entropy: Predicting Functional Correctness in LLM-Generated Code with Uncertainty Quantification
Dylan Bouchard, Mohit Singh Chauhan, Zeya Ahmad +1
Large language models have shown impressive capabilities in code generation, yet they often produce functionally incorrect code. Uncertainty quantification (UQ) methods have emerge…
UQLM: A Python Package for Uncertainty Quantification in Large Language Models
Dylan Bouchard, Mohit Singh Chauhan, David Skarbrevik +3
Hallucinations, defined as instances where Large Language Models (LLMs) generate false or misleading content, pose a significant challenge that impacts the safety and trust of down…
Uncertainty Quantification for Language Models: A Suite of Black-Box, White-Box, LLM Judge, and Ensemble Scorers
Dylan Bouchard, Mohit Singh Chauhan
Hallucinations are a persistent problem with Large Language Models (LLMs). As these models become increasingly used in high-stakes domains, such as healthcare and finance, the need…