TweepFake: about Detecting Deepfake Tweets

Abstract

The recent advances in language modeling significantly improved the generative capabilities of deep neural models: in 2019 OpenAI released GPT-2, a pre-trained language model that can autonomously generate coherent, non-trivial and human-like text samples. Since then, ever more powerful text generative models have been developed. Adversaries can exploit these tremendous generative capabilities to enhance social bots that will have the ability to write plausible deepfake messages, hoping to contaminate public debate. To prevent this, it is crucial to develop deepfake social media messages detection systems. However, to the best of our knowledge no one has ever addressed the detection of machine-generated texts on social networks like Twitter or Facebook. With the aim of helping the research in this detection field, we collected the first dataset of \real deepfake tweets, TweepFake. It is real in the sense that each deepfake tweet was actually posted on Twitter. We collected tweets from a total of 23 bots, imitating 17 human accounts. The bots are based on various generation techniques, i.e., Markov Chains, RNN, RNN+Markov, LSTM, GPT-2. We also randomly selected tweets from the humans imitated by the bots to have an overall balanced dataset of 25,572 tweets (half human and half bots generated). The dataset is publicly available on Kaggle. Lastly, we evaluated 13 deepfake text detection methods (based on various state-of-the-art approaches) to both demonstrate the challenges that Tweepfake poses and create a solid baseline of detection techniques. We hope that TweepFake can offer the opportunity to tackle the deepfake detection on social media messages as well.

References in corpus (12)

• Machine Learning in Automated Text Categorization
• Character-level Convolutional Networks for Text Classification
• API design for machine learning software: experiences from the scikit-learn project
• CTRL: A Conditional Transformer Language Model for Controllable Generation
• Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
• The paradigm-shift of social spambots: Evidence, theories, and tools for the arms race
• Release Strategies and the Social Impacts of Language Models
• Unpacking the Social Media Bot: A Typology to Guide Research and Policy
• TweepFake: about Detecting Deepfake Tweets
• Real or Fake? Learning to Discriminate Machine from Human Generated Text
• Attacking Neural Text Detectors
• On-the-fly Detection of Autogenerated Tweets

Cited by in corpus (17)

• Combining EfficientNet and Vision Transformers for Video Deepfake Detection
• TweepFake: about Detecting Deepfake Tweets
• Anatomy of an AI-powered malicious social botnet
• Identifying and Mitigating the Security Risks of Generative AI
• Deepfake: Definitions, Performance Metrics and Standards, Datasets and Benchmarks, and a Meta-Review
• Cross-Forgery Analysis of Vision Transformers and CNNs for Deepfake Image Detection
• Deep Learning Detection Method for Large Language Models-Generated Scientific Content
• Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts
• Artificial Text Detection via Examining the Topology of Attention Maps
• Adversarial Robustness of Neural-Statistical Features in Detection of Generative Transformers
• Findings of the The RuATD Shared Task 2022 on Artificial Text Detection in Russian
• Stylometry recognizes human and LLM-generated texts in short samples
• Detecting AI-Generated Texts in Cross-Domains
• APT-Pipe: A Prompt-Tuning Tool for Social Data Annotation using ChatGPT
• Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurement
• Forging the Forger: An Attempt to Improve Authorship Verification via Data Augmentation
• Seeing Isn't Believing: Addressing the Societal Impact of Deepfakes in Low-Tech Environments