Hallucination Detection in Foundation Models for Decision-Making: A Flexible Definition and Review of the State of the Art

Abstract

Autonomous systems are soon to be ubiquitous, spanning manufacturing, agriculture, healthcare, entertainment, and other industries. Most of these systems are developed with modular sub-components for decision-making, planning, and control that may be hand-engineered or learning-based. While these approaches perform well under the situations they were specifically designed for, they can perform especially poorly in out-of-distribution scenarios that will undoubtedly arise at test-time. The rise of foundation models trained on multiple tasks with impressively large datasets has led researchers to believe that these models may provide "common sense" reasoning that existing planners are missing, bridging the gap between algorithm development and deployment. While researchers have shown promising results in deploying foundation models to decision-making tasks, these models are known to hallucinate and generate decisions that may sound reasonable, but are in fact poor. We argue there is a need to step back and simultaneously design systems that can quantify the certainty of a model's decision, and detect when it may be hallucinating. In this work, we discuss the current use cases of foundation models for decision-making tasks, provide a general definition for hallucinations with examples, discuss existing approaches to hallucination detection and mitigation with a focus on decision problems, present guidelines, and explore areas for further research in this exciting field.

Accepted to ACM Computing Surveys; 55 pages, 5 tables, 3 figures

References in corpus (24)

• LLaMA: Open and Efficient Foundation Language Models
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs
• Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
• A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT
• Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond
• Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback
• LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples
• Cognitive Mirage: A Review of Hallucinations in Large Language Models
• Deploying and Evaluating LLMs to Program Service Mobile Robots
• A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation
• Hallucination of Multimodal Large Language Models: A Survey
• Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies
• DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model
• Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs
• Can you text what is happening? Integrating pre-trained language encoders into trajectory prediction models for autonomous driving
• Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Models
• Halo: Estimation and Reduction of Hallucinations in Open-Source Weak Large Language Models
• PURR: Efficiently Editing Language Model Hallucinations by Denoising Language Model Corruptions
• Introspective Tips: Large Language Model for In-Context Decision Making
• Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguity
• Sora Detector: A Unified Hallucination Detection for Large Text-to-Video Models
• Addressing Image Hallucination in Text-to-Image Generation through Factual Image Retrieval
• A Moral Imperative: The Need for Continual Superalignment of Large Language Models