Examining Racial Bias in an Online Abuse Corpus with Structural Topic Modeling
arXiv:2005.13041
Abstract
We use structural topic modeling to examine racial bias in data collected to train models to detect hate speech and abusive language in social media posts. We augment the abusive language dataset by adding an additional feature indicating the predicted probability of the tweet being written in African-American English. We then use structural topic modeling to examine the content of the tweets and how the prevalence of different topics is related to both abusiveness annotation and dialect prediction. We find that certain topics are disproportionately racialized and considered abusive. We discuss how topic modeling may be a useful approach for identifying bias in annotated data.
arXiv commentPlease cite the published version, see proceedings of ICWSM 2020
References in corpus (4)
- Semantics derived automatically from language corpora contain human-like biases
- Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
- Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes
- Racial Disparity in Natural Language Processing: A Case Study of Social Media African-American English