A Survey on Multimodal Disinformation Detection
arXiv:2103.12541
Abstract
Recent years have witnessed the proliferation of offensive content online such as fake news, propaganda, misinformation, and disinformation. While initially this was mostly about textual content, over time images and videos gained popularity, as they are much easier to consume, attract more attention, and spread further than text. As a result, researchers started leveraging different modalities and combinations thereof to tackle online multimodal offensive content. In this study, we offer a survey on the state-of-the-art on multimodal disinformation detection covering various combinations of modalities: text, images, speech, video, social media network structure, and temporal information. Moreover, while some studies focused on factuality, others investigated how harmful the content is. While these two components in the definition of disinformation (i) factuality, and (ii) harmfulness, are equally important, they are typically studied in isolation. Thus, we argue for the need to tackle disinformation detection by taking into account multiple modalities as well as both factuality and harmfulness, in the same framework. Finally, we discuss current challenges and future research directions
arXiv commentAccepted at COLING-2022, disinformation, misinformation, factuality, harmfulness, fake news, propaganda, multimodality, text, images, videos, network structure, temporality
References in corpus (20)
- VisualBERT: A Simple and Performant Baseline for Vision and Language
- FANG: Leveraging Social Context for Fake News Detection Using Graph Representation
- TI-CNN: Convolutional Neural Networks for Fake News Detection
- Supervised Multimodal Bitransformers for Classifying Images and Text
- The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes
- False Information on Web and Social Media: A Survey
- SAFE: Similarity-Aware Multi-Modal Fake News Detection
- Prediction of Cyberbullying Incidents on the Instagram Social Network
- Cyberbullying Detection in Social Networks Using Deep Learning Based Models; A Reproducibility Study
- Characterizing and Detecting Hateful Users on Twitter
- Gated Multimodal Units for Information Fusion
- Detecting Hate Speech in Multi-modal Memes
- Exploiting Multi-domain Visual Information for Fake News Detection
- Who's in the Gang? Revealing Coordinating Communities in Social Media
- Hierarchical Propagation Networks for Fake News Detection: Investigation and Exploitation
- Interpretable Multi-Modal Hate Speech Detection
- Network-based Fake News Detection: A Pattern-driven Approach
- Multilingual Multimodal Digital Deception Detection and Disinformation Spread across Social Platforms
- Two-path Deep Semi-supervised Learning for Timely Fake News Detection
- Predicting the Leading Political Ideology of YouTube Channels Using Acoustic, Textual, and Metadata Information