'Beating the news' with EMBERS: Forecasting Civil Unrest using Open Source Indicators

Abstract

We describe the design, implementation, and evaluation of EMBERS, an automated, 24x7 continuous system for forecasting civil unrest across 10 countries of Latin America using open source indicators such as tweets, news sources, blogs, economic indicators, and other data sources. Unlike retrospective studies, EMBERS has been making forecasts into the future since Nov 2012 which have been (and continue to be) evaluated by an independent T&E team (MITRE). Of note, EMBERS has successfully forecast the uptick and downtick of incidents during the June 2013 protests in Brazil. We outline the system architecture of EMBERS, individual models that leverage specific data sources, and a fusion and suppression engine that supports trading off specific evaluation criteria. EMBERS also provides an audit trail interface that enables the investigation of why specific predictions were made along with the data utilized for forecasting. Through numerous evaluations, we demonstrate the superiority of EMBERS over baserate methods and its capability to forecast significant societal happenings.

References in corpus (3)

• Probabilistic Similarity Logic
• A Universal Model of Global Civil Unrest
• Predicting Crowd Behavior with Big Public Data

Cited by in corpus (16)

• The dynamic of information-driven coordination phenomena: a transfer entropy analysis
• Crowdsourcing Cybersecurity: Cyber Attack Detection using Social Media
• Processing Social Media Messages in Mass Emergency: A Survey
• Using Social Media to Predict the Future: A Systematic Literature Review
• Forecasting Cyber Attacks with Imbalanced Data Sets and Different Time Granularities
• Characterizing Diseases from Unstructured Text: A Vocabulary Driven Word2vec Approach
• Extracting actionable information from microtexts
• Modeling Precursors for Event Forecasting via Nested Multi-Instance Learning
• Sparse Estimation of Multivariate Poisson Log-Normal Models from Count Data
• Inferring the geographic focus of online documents from social media sharing patterns
• Guided Deep List: Automating the Generation of Epidemiological Line Lists from Open Sources
• From Symbols to Embeddings: A Tale of Two Representations in Computational Social Science
• A Generic Framework for Interesting Subspace Cluster Detection in Multi-attributed Networks
• A Computational Framework for Multi-Modal Social Action Identification
• Can Self-Censorship in News Media be Detected Algorithmically? A Case Study in Latin America
• An Event Network for Exploring Open Information