Using Social Media to Predict the Future: A Systematic Literature Review

Abstract

Social media (SM) data provides a vast record of humanity's everyday thoughts, feelings, and actions at a resolution previously unimaginable. Because user behavior on SM is a reflection of events in the real world, researchers have realized they can use SM in order to forecast, making predictions about the future. The advantage of SM data is its relative ease of acquisition, large quantity, and ability to capture socially relevant information, which may be difficult to gather from other data sources. Promising results exist across a wide variety of domains, but one will find little consensus regarding best practices in either methodology or evaluation. In this systematic review, we examine relevant literature over the past decade, tabulate mixed results across a number of scientific disciplines, and identify common pitfalls and best practices. We find that SM forecasting is limited by data biases, noisy data, lack of generalizable results, a lack of domain-specific theory, and underlying complexity in many prediction tasks. But despite these shortcomings, recurring findings and promising results continue to galvanize researchers and demand continued investigation. Based on the existing literature, we identify research practices which lead to success, citing specific examples in each case and making recommendations for best practices. These recommendations will help researchers take advantage of the exciting possibilities offered by SM platforms.

References in corpus (10)

• Twitter mood predicts the stock market
• The Dynamics of Viral Marketing
• A High-Resolution Human Contact Network for Infectious Disease Transmission
• Geotagging One Hundred Million Twitter Accounts with Total Variation Minimization
• "I Wanted to Predict Elections with Twitter and all I got was this Lousy Paper" -- A Balanced Survey on Election Prediction using Twitter Data
• A Survey of Prediction Using Social Media
• Predicting Crowd Behavior with Big Public Data
• Using User Generated Online Photos to Estimate and Monitor Air Pollution in Major Cities
• The effect of wording on message propagation: Topic- and author-controlled natural experiments on Twitter
• 'Beating the news' with EMBERS: Forecasting Civil Unrest using Open Source Indicators

Cited by in corpus (7)

• Predictive Analysis on Twitter: Techniques and Applications
• Recurrent Event Network: Autoregressive Structure Inference over Temporal Knowledge Graphs
• Forecasting elections results via the voter model with stubborn nodes
• Temporal Knowledge Graph Reasoning Based on Evolutional Representation Learning
• Search from History and Reason for Future: Two-stage Reasoning on Temporal Knowledge Graphs
• ForecastQA: A Question Answering Challenge for Event Forecasting with Temporal Text Data
• TED-On: A Total Error Framework for Digital Traces of Human Behavior on Online Platforms