Classification of Multimodal Hate Speech -- The Winning Solution of Hateful Memes Challenge
arXiv:2012.01002
Abstract
Hateful Memes is a new challenge set for multimodal classification, focusing on detecting hate speech in multimodal memes. Difficult examples are added to the dataset to make it hard to rely on unimodal signals, which means only multimodal models can succeed. According to Kiela,the state-of-the-art methods perform poorly compared to humans (64.73% vs. 84.7% accuracy) on Hateful Memes. I propose a new model that combined multimodal with rules, which achieve the first ranking of accuracy and AUROC of 86.8% and 0.923 respectively. These rules are extracted from training set, and focus on improving the classification accuracy of difficult samples.
References in corpus (6)
- VisualBERT: A Simple and Performant Baseline for Vision and Language
- ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
- VL-BERT: Pre-training of Generic Visual-Linguistic Representations
- Learning Video Representations using Contrastive Bidirectional Transformer
- LXMERT: Learning Cross-Modality Encoder Representations from Transformers
- The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes