Robust Training of Social Media Image Classification Models for Rapid Disaster Response

Abstract

Images shared on social media help crisis managers gain situational awareness and assess incurred damages, among other response tasks. As the volume and velocity of such content are typically high, real-time image classification has become an urgent need for a faster disaster response. Recent advances in computer vision and deep neural networks have enabled the development of models for real-time image classification for a number of tasks, including detecting crisis incidents, filtering irrelevant images, classifying images into specific humanitarian categories, and assessing the severity of the damage. To develop robust real-time models, it is necessary to understand the capability of the publicly available pre-trained models for these tasks, which remains to be under-explored in the crisis informatics literature. In this study, we address such limitations by investigating ten different network architectures for four different tasks using the largest publicly available datasets for these tasks. We also explore various data augmentation strategies, semi-supervised techniques, and a multitask learning setup. In our extensive experiments, we achieve promising results.

arXiv commentSocial media images, Image Classification, Natural disasters, Crisis Informatics, Deep learning. Extended version of arXiv:2011.08916. arXiv admin note: substantial text overlap with arXiv:2011.08916

References in corpus (16)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Unsupervised Data Augmentation for Consistency Training
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Temporal Ensembling for Semi-Supervised Learning
• Interpolation Consistency Training for Semi-Supervised Learning
• MixMatch: A Holistic Approach to Semi-Supervised Learning
• ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
• Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFace
• Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response
• Automatic Image Filtering on Social Networks Using Deep Learning and Perceptual Hashing During Crises
• Rapid Damage Assessment Using Social Media Images by Combining Human and Machine Intelligence
• Graph Based Semi-supervised Learning with Convolution Neural Networks to Classify Crisis Related Tweets
• Localizing and Quantifying Damage in Social Media Images
• When Semi-Supervised Learning Meets Transfer Learning: Training Strategies, Models and Datasets