On the attribution of confidence to large language models

Abstract

Credences are mental states corresponding to degrees of confidence in propositions. Attribution of credences to Large Language Models (LLMs) is commonplace in the empirical literature on LLM evaluation. Yet the theoretical basis for LLM credence attribution is unclear. We defend three claims. First, our semantic claim is that LLM credence attributions are (at least in general) correctly interpreted literally, as expressing truth-apt beliefs on the part of scientists that purport to describe facts about LLM credences. Second, our metaphysical claim is that the existence of LLM credences is at least plausible, although current evidence is inconclusive. Third, our epistemic claim is that LLM credence attributions made in the empirical literature on LLM evaluation are subject to non-trivial sceptical concerns. It is a distinct possibility that even if LLMs have credences, LLM credence attributions are generally false because the experimental techniques used to assess LLM credences are not truth-tracking.

arXiv comment22 pages, 0 figures

References in corpus (26)

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• On the Opportunities and Risks of Foundation Models
• Sparks of Artificial General Intelligence: Early experiments with GPT-4
• Gemini: A Family of Highly Capable Multimodal Models
• Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
• Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
• A Survey on Evaluation of Large Language Models
• Consciousness in Artificial Intelligence: Insights from the Science of Consciousness
• Language Models (Mostly) Know What They Know
• Holistic Evaluation of Language Models
• Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs
• Teaching Models to Express Their Uncertainty in Words
• Bringing the People Back In: Contesting Benchmark Machine Learning Datasets
• Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation
• Sociotechnical Safety Evaluation of Generative AI Systems
• Towards Evaluating AI Systems for Moral Status Using Self-Reports
• A General Language Assistant as a Laboratory for Alignment
• Evaluating the Moral Beliefs Encoded in LLMs
• Standards for Belief Representations in LLMs
• Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examples
• Factual Confidence of LLMs: on Reliability and Robustness of Current Estimators
• Benchmarking LLMs via Uncertainty Quantification
• A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI
• Truthful AI: Developing and governing AI that does not lie
• Exploring the Nexus of Large Language Models and Legal Systems: A Short Survey