Fact-checking information from large language models can decrease headline discernment

Abstract

Fact checking can be an effective strategy against misinformation, but its implementation at scale is impeded by the overwhelming volume of information online. Recent artificial intelligence (AI) language models have shown impressive ability in fact-checking tasks, but how humans interact with fact-checking information provided by these models is unclear. Here, we investigate the impact of fact-checking information generated by a popular large language model (LLM) on belief in, and sharing intent of, political news headlines in a preregistered randomized control experiment. Although the LLM accurately identifies most false headlines (90%), we find that this information does not significantly improve participants' ability to discern headline accuracy or share accurate news. In contrast, viewing human-generated fact checks enhances discernment in both cases. Subsequent analysis reveals that the AI fact-checker is harmful in specific cases: it decreases beliefs in true headlines that it mislabels as false and increases beliefs in false headlines that it is unsure about. On the positive side, AI fact-checking information increases the sharing intent for correctly labeled true headlines. When participants are given the option to view LLM fact checks and choose to do so, they are significantly more likely to share both true and false news but only more likely to believe false headlines. Our findings highlight an important source of potential harm stemming from AI applications and underscore the critical need for policies to prevent or mitigate such unintended consequences.

References in corpus (18)

• The science of fake news
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Language Models are Few-Shot Learners
• A Survey of Fake News: Fundamental Theories, Detection Methods, and Opportunities
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making
• Computational fact checking from knowledge networks
• Release Strategies and the Social Impacts of Language Models
• AI model GPT-3 (dis)informs us better than humans
• A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models
• "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust
• The effect of source disclosure on evaluation of AI-generated messages: A two-part study
• The Perils & Promises of Fact-checking with Large Language Models
• Anatomy of an AI-powered malicious social botnet
• Fact-checking information from large language models can decrease headline discernment
• Customized large language models can outperform Community Notes in correcting misinformation
• Scalable Fact-checking with Human-in-the-Loop
• The Effects of Interactive AI Design on User Behavior: An Eye-tracking Study of Fact-checking COVID-19 Claims
• In Generative AI we Trust: Can Chatbots Effectively Verify Political Information?

Cited by in corpus (3)

• Fact-checking information from large language models can decrease headline discernment
• Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Models
• Accuracy and Political Bias of News Source Credibility Ratings by Large Language Models