Controlled Analyses of Social Biases in Wikipedia Bios
arXiv:2101.00078 · doi:10.1145/3485447.3512134
Abstract
Social biases on Wikipedia, a widely-read global platform, could greatly influence public opinion. While prior research has examined man/woman gender bias in biography articles, possible influences of other demographic attributes limit conclusions. In this work, we present a methodology for analyzing Wikipedia pages about people that isolates dimensions of interest (e.g., gender), from other attributes (e.g., occupation). Given a target corpus for analysis (e.g.~biographies about women), we present a method for constructing a comparison corpus that matches the target corpus in as many attributes as possible, except the target one. We develop evaluation metrics to measure how well the comparison corpus aligns with the target corpus and then examine how articles about gender and racial minorities (cis. women, non-binary people, transgender women, and transgender men; African American, Asian American, and Hispanic/Latinx American people) differ from other articles. In addition to identifying suspect social biases, our results show that failing to control for covariates can result in different conclusions and veil biases. Our contributions include methodology that facilitates further analyses of bias in Wikipedia articles, findings that can aid Wikipedia editors in reducing biases, and a framework and evaluation metrics to guide future work in this area.
Accepted to the Web Conference 2022 (WWW '22)
References in corpus (6)
- Matching Methods for Causal Inference: A Review and a Look Forward
- Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
- Towards a Critical Race Methodology in Algorithmic Fairness
- The Tower of Babel Meets Web 2.0: User-Generated Content and its Applications in a Multilingual Context
- Interactions of cultures and top people of Wikipedia from ranking of 24 language editions
- An Interpretability Illusion for BERT