Fine-tuning of Pre-trained Transformers for Hate, Offensive, and Profane Content Detection in English and Marathi
arXiv:2110.12687
Abstract
This paper describes neural models developed for the Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages Shared Task 2021. Our team called neuro-utmn-thales participated in two tasks on binary and fine-grained classification of English tweets that contain hate, offensive, and profane content (English Subtasks A & B) and one task on identification of problematic content in Marathi (Marathi Subtask A). For English subtasks, we investigate the impact of additional corpora for hate speech detection to fine-tune transformer models. We also apply a one-vs-rest approach based on Twitter-RoBERTa to discrimination between hate, profane and offensive posts. Our models ranked third in English Subtask A with the F1-score of 81.99% and ranked second in English Subtask B with the F1-score of 65.77%. For the Marathi tasks, we propose a system based on the Language-Agnostic BERT Sentence Embedding (LaBSE). This model achieved the second result in Marathi Subtask A obtaining an F1 of 88.08%.
Accepted for FIRE'21: Forum for Information Retrieval Evaluation 2021
References in corpus (4)
- Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books
- Automated Hate Speech Detection and the Problem of Offensive Language
- SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)
- Cross-lingual Offensive Language Identification for Low Resource Languages: The Case of Marathi