Cross-lingual Zero- and Few-shot Hate Speech Detection Utilising Frozen Transformer Language Models and AXEL

Abstract

Detecting hate speech, especially in low-resource languages, is a non-trivial challenge. To tackle this, we developed a tailored architecture based on frozen, pre-trained Transformers to examine cross-lingual zero-shot and few-shot learning, in addition to uni-lingual learning, on the HatEval challenge data set. With our novel attention-based classification block AXEL, we demonstrate highly competitive results on the English and Spanish subsets. We also re-sample the English subset, enabling additional, meaningful comparisons in the future.

References in corpus (7)

• Cross-lingual Language Model Pretraining
• Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
• Word Translation Without Parallel Data
• Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
• Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT
• MAMNet: Multi-path Adaptive Modulation Network for Image Super-Resolution
• Dissecting Contextual Word Embeddings: Architecture and Representation

Cited by in corpus (10)

• MM-COVID: A Multilingual and Multimodal Data Repository for Combating COVID-19 Disinformation
• Moderating New Waves of Online Hate with Chain-of-Thought Reasoning in Large Language Models
• Bayesian Methods for Semi-supervised Text Annotation
• A Neighbourhood Framework for Resource-Lean Content Flagging
• Multilingual Auxiliary Tasks Training: Bridging the Gap between Languages for Zero-Shot Transfer of Hate Speech Detection Models
• Transfer Learning for Multi-lingual Tasks -- a Survey
• Cross-lingual hate speech detection based on multilingual domain-specific word embeddings
• The MuSe 2021 Multimodal Sentiment Analysis Challenge: Sentiment, Emotion, Physiological-Emotion, and Stress
• Cross-lingual COVID-19 Fake News Detection
• Multilingual Code-Switching for Zero-Shot Cross-Lingual Intent Prediction and Slot Filling