Verifying the Robustness of Automatic Credibility Assessment

Abstract

Text classification methods have been widely investigated as a way to detect content of low credibility: fake news, social media bots, propaganda, etc. Quite accurate models (likely based on deep neural networks) help in moderating public electronic platforms and often cause content creators to face rejection of their submissions or removal of already published texts. Having the incentive to evade further detection, content creators try to come up with a slightly modified version of the text (known as an attack with an adversarial example) that exploit the weaknesses of classifiers and result in a different output. Here we systematically test the robustness of common text classifiers against available attacking techniques and discover that, indeed, meaning-preserving changes in input text can mislead the models. The approaches we test focus on finding vulnerable spans in text and replacing individual characters or words, taking into account the similarity between the original and replacement content. We also introduce BODEGA: a benchmark for testing both victim models and attack methods on four misinformation detection tasks in an evaluation framework designed to simulate real use-cases of content moderation. The attacked tasks include (1) fact checking and detection of (2) hyperpartisan news, (3) propaganda and (4) rumours. Our experimental results show that modern large language models are often more vulnerable to attacks than previous, smaller solutions, e.g. attacks on GEMMA being up to 27\% more successful than those on BERT. Finally, we manually analyse a subset adversarial examples and check what kinds of modifications are used in successful attacks.

References in corpus (10)

• A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly
• Gemma: Open Models Based on Gemini Research and Technology
• Word-level Textual Adversarial Attacking as Combinatorial Optimization
• Fake News Detection via NLP is Vulnerable to Adversarial Attacks
• OpenAttack: An Open-source Textual Adversarial Attack Toolkit
• SoK: Content Moderation in Social Media, from Guidelines to Enforcement, and Research to Practice
• Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures
• Technology-Enabled Disinformation: Summary, Lessons, and Recommendations
• Neural Language Model Based Training Data Augmentation for Weakly Supervised Early Rumor Detection
• Robustness Over Time: Understanding Adversarial Examples' Effectiveness on Longitudinal Versions of Large Language Models

Cited by in corpus (1)

• A Survey on Automatic Credibility Assessment Using Textual Credibility Signals in the Era of Large Language Models