Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech

Abstract

Recent studies have alarmed that many online hate speeches are implicit. With its subtle nature, the explainability of the detection of such hateful speech has been a challenging problem. In this work, we examine whether ChatGPT can be used for providing natural language explanations (NLEs) for implicit hateful speech detection. We design our prompt to elicit concise ChatGPT-generated NLEs and conduct user studies to evaluate their qualities by comparison with human-written NLEs. We discuss the potential and limitations of ChatGPT in the context of implicit hateful speech research.

Accepted by The Web Conference Companion, 2023

References in corpus (1)

• Chain of Explanation: New Prompting Method to Generate Higher Quality Natural Language Explanation for Implicit Hate Speech

Cited by in corpus (18)

• ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks
• Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
• Decoding ChatGPT: A Taxonomy of Existing Research, Current Challenges, and Possible Future Directions
• The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships
• A Large Language Model Approach to Educational Survey Feedback Analysis
• CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation
• If in a Crowdsourced Data Annotation Pipeline, a GPT-4
• Embedding Democratic Values into Social Media AIs via Societal Objective Functions
• AI-induced sexual harassment: Investigating Contextual Characteristics and User Reactions of Sexual Harassment by a Companion Chatbot
• LLM Teacher-Student Framework for Text Classification With No Manually Annotated Data: A Case Study in IPTC News Topic Classification
• Discrete Prompt Compression with Reinforcement Learning
• OffensiveLang: A Community Based Implicit Offensive Language Dataset
• GPT Assisted Annotation of Rhetorical and Linguistic Features for Interpretable Propaganda Technique Detection in News Text
• Using language models to label clusters of scientific documents
• Specializing General-purpose LLM Embeddings for Implicit Hate Speech Detection across Datasets
• LLM-Supported Content Analysis of Motivated Reasoning on Climate Change
• Reliable Annotations with Less Effort: Evaluating LLM-Human Collaboration in Search Clarifications
• DiVa: An Iterative Framework to Harvest More Diverse and Valid Labels from User Comments for Music