Enhance Multimodal Transformer With External Label And In-Domain Pretrain: Hateful Meme Challenge Winning Solution
arXiv:2012.08290
Abstract
Hateful meme detection is a new research area recently brought out that requires both visual, linguistic understanding of the meme and some background knowledge to performing well on the task. This technical report summarises the first place solution of the Hateful Meme Detection Challenge 2020, which extending state-of-the-art visual-linguistic transformers to tackle this problem. At the end of the report, we also point out the shortcomings and possible directions for improving the current methodology.
References in corpus (8)
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
- Large-Scale Adversarial Training for Vision-and-Language Representation Learning
- The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes
- FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age
- K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters
- ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graph
- Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering
Cited by in corpus (8)
- Disentangling Hate in Online Memes
- Model Sketching: Centering Concepts in Early-Stage Machine Learning Model Design
- PromptMTopic: Unsupervised Multimodal Topic Modeling of Memes using Large Language Models
- Toxic Memes: A Survey of Computational Perspectives on the Detection and Explanation of Meme Toxicities
- Meme Sentiment Analysis Enhanced with Multimodal Spatial Encoding and Facial Embedding
- Towards A Multi-agent System for Online Hate Speech Detection
- What is Multimodality?
- Detecting Harmful Memes and Their Targets