Interactive Learning for Identifying Relevant Tweets to Support Real-time Situational Awareness

Abstract

Various domain users are increasingly leveraging real-time social media data to gain rapid situational awareness. However, due to the high noise in the deluge of data, effectively determining semantically relevant information can be difficult, further complicated by the changing definition of relevancy by each end user for different events. The majority of existing methods for short text relevance classification fail to incorporate users' knowledge into the classification process. Existing methods that incorporate interactive user feedback focus on historical datasets. Therefore, classifiers cannot be interactively retrained for specific events or user-dependent needs in real-time. This limits real-time situational awareness, as streaming data that is incorrectly classified cannot be corrected immediately, permitting the possibility for important incoming data to be incorrectly classified as well. We present a novel interactive learning framework to improve the classification process in which the user iteratively corrects the relevancy of tweets in real-time to train the classification model on-the-fly for immediate predictive improvements. We computationally evaluate our classification model adapted to learn at interactive rates. Our results show that our approach outperforms state-of-the-art machine learning models. In addition, we integrate our framework with the extended Social Media Analytics and Reporting Toolkit (SMART) 2.0 system, allowing the use of our interactive learning framework within a visual analytics system tailored for real-time situational awareness. To demonstrate our framework's effectiveness, we provide domain expert feedback from first responders who used the extended SMART 2.0 system.

12 pages, 8 figures, 3 tables, IEEE VIS VAST 2019, TVCG

References in corpus (5)

• Distributed Representations of Words and Phrases and their Compositionality
• Comparative Study of CNN and RNN for Natural Language Processing
• Twitter as a Lifeline: Human-annotated Twitter Corpora for NLP of Crisis-related Messages
• Applications of Online Deep Learning for Crisis Response Using Social Media Information
• Rapid Classification of Crisis-Related Data on Social Networks using Convolutional Neural Networks

Cited by in corpus (5)

• City-level Geolocation of Tweets for Real-time Visual Analytics
• A Survey of Visual Analytics Techniques for Machine Learning
• Exploring Descriptions of Movement Through Geovisual Analytics
• GeoDEN: A Visual Exploration Tool for Analysing the Geographic Spread of Dengue Serotypes
• Geovisual Analytics and Interactive Machine Learning for Situational Awareness