Detecting Harmful Memes and Their Targets
arXiv:2110.00413
Abstract
Among the various modes of communication in social media, the use of Internet memes has emerged as a powerful means to convey political, psychological, and socio-cultural opinions. Although memes are typically humorous in nature, recent days have witnessed a proliferation of harmful memes targeted to abuse various social entities. As most harmful memes are highly satirical and abstruse without appropriate contexts, off-the-shelf multimodal models may not be adequate to understand their underlying semantics. In this work, we propose two novel problem formulations: detecting harmful memes and the social entities that these harmful memes target. To this end, we present HarMeme, the first benchmark dataset, containing 3,544 memes related to COVID-19. Each meme went through a rigorous two-stage annotation process. In the first stage, we labeled a meme as very harmful, partially harmful, or harmless; in the second stage, we further annotated the type of target(s) that each harmful meme points to: individual, organization, community, or society/general public/other. The evaluation results using ten unimodal and multimodal models highlight the importance of using multimodal signals for both tasks. We further discuss the limitations of these models and we argue that more research is needed to address these problems.
harmful memes, multimodality, social media
References in corpus (12)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Hate Speech in Pixels: Detection of Offensive Memes towards Automatic Moderation
- Enhance Multimodal Transformer With External Label And In-Domain Pretrain: Hateful Meme Challenge Winning Solution
- Vilio: State-of-the-art Visio-Linguistic Models applied to Hateful Memes
- Detecting Hate Speech in Multi-modal Memes
- Are we pretraining it right? Digging deeper into visio-linguistic pretraining
- Detecting Hateful Memes Using a Multimodal Deep Ensemble
- Multimodal Classification for Analysing Social Media
- Classification of Multimodal Hate Speech -- The Winning Solution of Hateful Memes Challenge
- Dissecting the Meme Magic: Understanding Indicators of Virality in Image Memes
- Exercise? I thought you said 'Extra Fries': Leveraging Sentence Demarcations and Multi-hop Attention for Meme Affect Analysis
- Multimodal Learning for Hateful Memes Detection