VEWS: A Wikipedia Vandal Early Warning System

Abstract

We study the problem of detecting vandals on Wikipedia before any human or known vandalism detection system reports flagging potential vandals so that such users can be presented early to Wikipedia administrators. We leverage multiple classical ML approaches, but develop 3 novel sets of features. Our Wikipedia Vandal Behavior (WVB) approach uses a novel set of user editing patterns as features to classify some users as vandals. Our Wikipedia Transition Probability Matrix (WTPM) approach uses a set of features derived from a transition probability matrix and then reduces it via a neural net auto-encoder to classify some users as vandals. The VEWS approach merges the previous two approaches. Without using any information (e.g. reverts) provided by other users, these algorithms each have over 85% classification accuracy. Moreover, when temporal recency is considered, accuracy goes to almost 90%. We carry out detailed experiments on a new data set we have created consisting of about 33K Wikipedia users (including both a black list and a white list of editors) and containing 770K edits. We describe specific behaviors that distinguish between vandals and non-vandals. We show that VEWS beats ClueBot NG and STiki, the best known algorithms today for vandalism detection. Moreover, VEWS detects far more vandals than ClueBot NG and on average, detects them 2.39 edits before ClueBot NG when both detect the vandal. However, we show that the combination of VEWS and ClueBot NG can give a fully automated vandal early warning system with even higher accuracy.

arXiv commentTo appear in Proceedings of the 21st ACM SIGKDD Conference of Knowledge Discovery and Data Mining (KDD 2015)

References in corpus (2)

• Scikit-learn: Machine Learning in Python
• Wikipedia Vandalism Detection Through Machine Learning: Feature Review and New Proposals: Lab Report for PAN at CLEF 2010

Cited by in corpus (21)

• Deep Learning for Anomaly Detection: A Review
• Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks
• Community Interaction and Conflict on the Web
• An Army of Me: Sockpuppets in Online Discussion Communities
• False Information on Web and Social Media: A Survey
• Rank List Sensitivity of Recommender Systems to Interaction Perturbations
• Learning Dynamic Embeddings from Temporal Interactions
• One-Class Adversarial Nets for Fraud Detection
• EnsemFDet: An Ensemble Approach to Fraud Detection based on Bipartite Graph
• Spectrum-based deep neural networks for fraud detection
• Wikipedia Vandal Early Detection: from User Behavior to User Embedding
• PETGEN: Personalized Text Generation Attack on Deep Sequence Embedding-based Classification Models
• Insider Threat Detection via Hierarchical Neural Temporal Point Processes
• Attention-Based Vandalism Detection in OpenStreetMap
• A preliminary approach to knowledge integrity risk assessment in Wikipedia projects
• Overview of the Wikidata Vandalism Detection Task at WSDM Cup 2017
• GAN Augmented Text Anomaly Detection with Sequences of Deep Statistics
• SNE: Signed Network Embedding
• Interpretable classification of wiki-review streams
• SAFE: A Neural Survival Analysis Model for Fraud Early Detection
• SGPT: Few-Shot Prompt Tuning for Signed Graphs