Defending Against Neural Fake News

Abstract

Recent progress in natural language generation has raised dual-use concerns. While applications like summarization and translation are positive, the underlying technology also might enable adversaries to generate neural fake news: targeted propaganda that closely mimics the style of real news. Modern computer security relies on careful threat modeling: identifying potential threats and vulnerabilities from an adversary's point of view, and exploring potential mitigations to these threats. Likewise, developing robust defenses against neural fake news requires us first to carefully investigate and characterize the risks of these models. We thus present a model for controllable text generation called Grover. Given a headline like `Link Found Between Vaccines and Autism,' Grover can generate the rest of the article; humans find these generations to be more trustworthy than human-written disinformation. Developing robust verification techniques against generators like Grover is critical. We find that best current discriminators can classify neural fake news from real, human-written, news with 73% accuracy, assuming access to a moderate level of training data. Counterintuitively, the best defense against Grover turns out to be Grover itself, with 92% accuracy, demonstrating the importance of public release of strong generators. We investigate these results further, showing that exposure bias -- and sampling strategies that alleviate its effects -- both leave artifacts that similar discriminators can pick up on. We conclude by discussing ethical issues regarding the technology, and plan to release Grover publicly, helping pave the way for better detection of neural fake news.

arXiv commentNeurIPS 2019 camera ready version. Project page/code/demo at https://rowanzellers.com/grover

References in corpus (10)

• The Curious Case of Neural Text Degeneration
• Exploring the Limits of Language Modeling
• Hoaxy: A Platform for Tracking Online Misinformation
• Automatic Detection of Fake News
• Release Strategies and the Social Impacts of Language Models
• Insertion Transformer: Flexible Sequence Generation via Insertion Operations
• Language GANs Falling Short
• Unifying Human and Statistical Evaluation for Natural Language Generation
• Insertion-based Decoding with automatically Inferred Generation Order
• Unsupervised Domain Adaptation of Contextualized Embeddings for Sequence Labeling

Cited by in corpus (37)

• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Language Models are Few-Shot Learners
• Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
• CTRL: A Conditional Transformer Language Model for Controllable Generation
• SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching
• ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training
• Collecting the Public Perception of AI and Robot Rights
• ProGen: Language Modeling for Protein Generation
• Forbidden knowledge in machine learning -- Reflections on the limits of research and publication
• GO FIGURE: A Meta Evaluation of Factuality in Summarization
• ESR: Ethics and Society Review of Artificial Intelligence Research
• Rethinking Automatic Evaluation in Sentence Simplification
• Temporal Common Sense Acquisition with Minimal Supervision
• Generating Sentiment-Preserving Fake Online Reviews Using Neural Language Models and Their Human- and Machine-based Detection
• The Offense-Defense Balance of Scientific Knowledge: Does Publishing AI Research Reduce Misuse?
• Improved Natural Language Generation via Loss Truncation
• Topic-Preserving Synthetic News Generation: An Adversarial Deep Reinforcement Learning Approach
• Importance-Aware Learning for Neural Headline Editing
• "Are you home alone?" "Yes" Disclosing Security and Privacy Vulnerabilities in Alexa Skills
• ColdGANs: Taming Language GANs with Cautious Sampling Strategies
• The Rumour Mill: Making the Spread of Misinformation Explicit and Tangible
• DLGNet: A Transformer-based Model for Dialogue Response Generation
• MALCOM: Generating Malicious Comments to Attack Neural Fake News Detection Models
• Training Optimus Prime, M.D.: Generating Medical Certification Items by Fine-Tuning OpenAI's gpt2 Transformer Model
• Attacking Neural Text Detectors
• Differential Privacy and Natural Language Processing to Generate Contextually Similar Decoy Messages in Honey Encryption Scheme
• How does Truth Evolve into Fake News? An Empirical Study of Fake News Evolution
• Adding A Filter Based on The Discriminator to Improve Unconditional Text Generation
• Reverse Engineering Configurations of Neural Text Generation Models
• A multi-layer approach to disinformation detection on Twitter
• The R-U-A-Robot Dataset: Helping Avoid Chatbot Deception by Detecting User Questions About Human or Non-Human Identity
• To Beam Or Not To Beam: That is a Question of Cooperation for Language GANs
• No News is Good News: A Critique of the One Billion Word Benchmark
• Graph Reasoning with Context-Aware Linearization for Interpretable Fact Extraction and Verification
• A Novel Corpus of Discourse Structure in Humans and Computers
• LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network
• Personalized Patent Claim Generation and Measurement