7 papers
Efficient Hallucination Detection for LLMs Using Uncertainty-Aware Attention Heads
Artem Vazhentsev, Lyudmila Rvanova, Gleb Kuzmin +8
While large language models (LLMs) have become highly capable, they remain prone to factual inaccuracies, commonly referred to as "hallucinations." Uncertainty quantification (UQ)…
Harnessing non-adversarial robustness in large language models
Qinghua Zhou, Ellina Aleshina, Andrey Lovyagin +6
The work presents an approach for addressing the challenge of robustness in Large Language Models (LLMs) to alterations and potential errors caused by semantically similar but text…
Evolutionary Search for Automated Design of Uncertainty Quantification Methods
Mikhail Seleznyov, Daniil Korbut, Viktor Moskvoretskii +3
Uncertainty quantification (UQ) methods for large language models are predominantly designed by hand based on domain knowledge and heuristics, limiting their scalability and genera…
Unconditional Truthfulness: Learning Unconditional Uncertainty of Large Language Models
Artem Vazhentsev, Ekaterina Fadeeva, Rui Xing +7
Uncertainty quantification (UQ) has emerged as a promising approach for detecting hallucinations and low-quality output of Large Language Models (LLMs). However, obtaining proper u…
Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph
Roman Vashurin, Ekaterina Fadeeva, Artem Vazhentsev +12
The rapid proliferation of large language models (LLMs) has stimulated researchers to seek effective and efficient approaches to deal with LLM hallucinations and low-quality output…
Token-Level Density-Based Uncertainty Quantification Methods for Eliciting Truthfulness of Large Language Models
Artem Vazhentsev, Lyudmila Rvanova, Ivan Lazichny +4
Uncertainty quantification (UQ) is a prominent approach for eliciting truthful answers from large language models (LLMs). To date, information-based and consistency-based UQ have b…