Deep Learning Models for Multilingual Hate Speech Detection
arXiv:2004.06465
Abstract
Hate speech detection is a challenging problem with most of the datasets available in only one language: English. In this paper, we conduct a large scale analysis of multilingual hate speech in 9 languages from 16 different sources. We observe that in low resource setting, simple models such as LASER embedding with logistic regression performs the best, while in high resource setting BERT based models perform better. In case of zero-shot classification, languages such as Italian and Portuguese achieve good results. Our proposed framework could be used as an efficient solution for low-resource languages. These models could also act as good baselines for future multilingual hate speech detection tasks. We have made our code and experimental settings public for other researchers at https://github.com/punyajoy/DE-LIMIT.
arXiv comment16 pages, Accepted at ECML-PKDD 2020
References in corpus (5)
- Efficient Estimation of Word Representations in Vector Space
- Word Translation Without Parallel Data
- Characterizing and Detecting Hateful Users on Twitter
- Hate Lingo: A Target-based Linguistic Analysis of Hate Speech in Social Media
- Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition
Cited by in corpus (10)
- Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective
- Hate begets Hate: A Temporal Study of Hate Speech
- What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus
- IIITT@LT-EDI-EACL2021-Hope Speech Detection: There is always Hope in Transformers
- Accenture at CheckThat! 2020: If you say so: Post-hoc fact-checking of claims using transformer-based models
- One to rule them all: Towards Joint Indic Language Hate Speech Detection
- Hate and Offensive Speech Detection in Hindi and Marathi
- Language-Driven Image Style Transfer
- Cross-lingual hate speech detection based on multilingual domain-specific word embeddings
- MIPT-NSU-UTMN at SemEval-2021 Task 5: Ensembling Learning with Pre-trained Language Models for Toxic Spans Detection