Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective

Abstract

The pervasiveness of abusive content on the internet can lead to severe psychological and physical harm. Significant effort in Natural Language Processing (NLP) research has been devoted to addressing this problem through abusive content detection and related sub-areas, such as the detection of hate speech, toxicity, cyberbullying, etc. Although current technologies achieve high classification performance in research studies, it has been observed that the real-life application of this technology can cause unintended harms, such as the silencing of under-represented groups. We review a large body of NLP research on automatic abuse detection with a new focus on ethical challenges, organized around eight established ethical principles: privacy, accountability, safety and security, transparency and explainability, fairness and non-discrimination, human control of technology, professional responsibility, and promotion of human values. In many cases, these principles relate not only to situational ethical codes, which may be context-dependent, but are in fact connected to universal human rights, such as the right to privacy, freedom from discrimination, and freedom of expression. We highlight the need to examine the broad social impacts of this technology, and to bring ethical and human rights considerations to every stage of the application life-cycle, from task formulation and dataset design, to model training and evaluation, to application deployment. Guided by these principles, we identify several opportunities for rights-respecting, socio-technical solutions to detect and confront online abuse, including `nudging', `quarantining', value sensitive design, counter-narratives, style transfer, and AI-driven public education applications.

arXiv commentpublished in Journal of Artificial Intelligence Research, 71: 431-478, July 2021

References in corpus (22)

• Semantics derived automatically from language corpora contain human-like biases
• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
• ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
• Federated Machine Learning: Concept and Applications
• Directions in Abusive Language Training Data: Garbage In, Garbage Out
• The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes
• All You Need is "Love": Evading Hate-speech Detection
• Conversations Gone Awry: Detecting Early Signs of Conversational Failure
• Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Bias
• Deep Learning Models for Multilingual Hate Speech Detection
• Context-Aware Attention for Understanding Twitter Abuse
• Detecting Online Hate Speech Using Context Aware Models
• HateCheck: Functional Tests for Hate Speech Detection Models
• Offensive Language and Hate Speech Detection for Danish
• Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition
• Measuring Bias in Contextualized Word Representations
• SWE2: SubWord Enriched and Significant Word Emphasized Framework for Hate Speech Detection
• Challenges in Automated Debiasing for Toxic Language Detection
• That Message Went Viral?! Exploratory Analytics and Sentiment Analysis into the Propagation of Tweets
• Annotating for Hate Speech: The MaNeCo Corpus and Some Input from Critical Discourse Analysis
• From Soft Classifiers to Hard Decisions: How fair can we be?
• On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning

Cited by in corpus (4)

• A Critical Reflection on the Use of Toxicity Detection Algorithms in Proactive Content Moderation Systems
• Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directions
• Cross-lingual Offensive Language Detection: A Systematic Review of Datasets, Transfer Approaches and Challenges
• Toxic comments reduce the activity of volunteer editors on Wikipedia