Exposing Influence Campaigns in the Age of LLMs: A Behavioral-Based AI Approach to Detecting State-Sponsored Trolls

Abstract

The detection of state-sponsored trolls operating in influence campaigns on social media is a critical and unsolved challenge for the research community, which has significant implications beyond the online realm. To address this challenge, we propose a new AI-based solution that identifies troll accounts solely through behavioral cues associated with their sequences of sharing activity, encompassing both their actions and the feedback they receive from others. Our approach does not incorporate any textual content shared and consists of two steps: First, we leverage an LSTM-based classifier to determine whether account sequences belong to a state-sponsored troll or an organic, legitimate user. Second, we employ the classified sequences to calculate a metric named the "Troll Score", quantifying the degree to which an account exhibits troll-like behavior. To assess the effectiveness of our method, we examine its performance in the context of the 2016 Russian interference campaign during the U.S. Presidential election. Our experiments yield compelling results, demonstrating that our approach can identify account sequences with an AUC close to 99% and accurately differentiate between Russian trolls and organic users with an AUC of 91%. Notably, our behavioral-based approach holds a significant advantage in the ever-evolving landscape, where textual and linguistic properties can be easily mimicked by Large Language Models (LLMs): In contrast to existing language-based techniques, it relies on more challenging-to-replicate behavioral cues, ensuring greater resilience in identifying influence campaigns, especially given the potential increase in the usage of LLMs for generating inauthentic content. Finally, we assessed the generalizability of our solution to various entities driving different information operations and found promising results that will guide future research.

22

References in corpus (7)

• Propaganda and Misinformation on Facebook and Twitter during the Russian Invasion of Ukraine
• One Year of COVID-19 Vaccine Misinformation on Twitter: Longitudinal Study
• Gender Classification and Bias Mitigation in Facial Images
• An Automated Pipeline for Character and Relationship Extraction from Readers' Literary Book Reviews on Goodreads.com
• How does Twitter account moderation work? Dynamics of account creation and suspension on Twitter during major geopolitical events
• Twitter Spam and False Accounts Prevalence, Detection and Characterization: A Survey
• Analysing user identity via time-sensitive semantic edit distance (t-SED): A case study of Russian trolls on Twitter

Cited by in corpus (4)

• A survey of textual cyber abuse detection using cutting-edge language models and large language models
• Exposing Cross-Platform Coordinated Inauthentic Activity in the Run-Up to the 2024 U.S. Election
• Unsupervised detection of coordinated information operations in the wild
• Network-informed Prompt Engineering against Organized Astroturf Campaigns under Extreme Class Imbalance