Vilio: State-of-the-art Visio-Linguistic Models applied to Hateful Memes
arXiv:2012.07788
Abstract
This work presents Vilio, an implementation of state-of-the-art visio-linguistic models and their application to the Hateful Memes Dataset. The implemented models have been fitted into a uniform code-base and altered to yield better performance. The goal of Vilio is to provide a user-friendly starting point for any visio-linguistic problem. An ensemble of 5 different V+L models implemented in Vilio achieves 2nd place in the Hateful Memes Challenge out of 3,300 participants. The code is available at https://github.com/Muennighoff/vilio.
arXiv commentPresented at NIPS 2020
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Language Models are Few-Shot Learners
- VisualBERT: A Simple and Performant Baseline for Vision and Language
- Averaging Weights Leads to Wider Optima and Better Generalization
- LXMERT: Learning Cross-Modality Encoder Representations from Transformers
- UNITER: UNiversal Image-TExt Representation Learning
- Pythia v0.1: the Winning Entry to the VQA Challenge 2018
- The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes
- ERNIE: Enhanced Language Representation with Informative Entities
- ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graph
- Multi-Sample Dropout for Accelerated Training and Better Generalization
- On the adequacy of untuned warmup for adaptive optimization
Cited by in corpus (9)
- Disentangling Hate in Online Memes
- Toxic Memes: A Survey of Computational Perspectives on the Detection and Explanation of Meme Toxicities
- HateProof: Are Hateful Meme Detection Systems really Robust?
- MATK: The Meme Analytical Tool Kit
- Towards A Multi-agent System for Online Hate Speech Detection
- Diagnosing the Impact of AI on Radiology in China
- MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets
- Caption Enriched Samples for Improving Hateful Memes Detection
- Detecting Harmful Memes and Their Targets