Detecting Hateful Memes Using a Multimodal Deep Ensemble

Abstract

While significant progress has been made using machine learning algorithms to detect hate speech, important technical challenges still remain to be solved in order to bring their performance closer to human accuracy. We investigate several of the most recent visual-linguistic Transformer architectures and propose improvements to increase their performance for this task. The proposed model outperforms the baselines by a large margin and ranks 5${}^{th}$ on the leaderboard out of 3,100+ participants.

6 pages, NeurIPS 2020, The Hateful Memes Challenge Workshop at NeurIPS 2020

References in corpus (2)

• Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challenge
• Are we pretraining it right? Digging deeper into visio-linguistic pretraining