most citedLimitations of Pinned AUC for Measuring Unintended Bias

8 citations · 12 across the 2 of their papers we have counts for

collaborators

5 papers

cs.CL2020

Toxicity Detection: Does Context Really Matter?

John Pavlopoulos, Jeffrey Sorensen, Lucas Dixon +2

Moderation is crucial to promoting healthy on-line discussions. Although several `toxicity' detection datasets and models have been published, most of them ignore the context of th…

cs.CL2020

Classifying Constructive Comments

Varada Kolhatkar, Nithum Thain, Jeffrey Sorensen +2

We introduce the Constructive Comments Corpus (C3), comprised of 12,000 annotated news comments, intended to help build new tools for online communities to improve the quality of t…

cs.LG2019

Debiasing Embeddings for Reduced Gender Bias in Text Classification

Flavien Prost, Nithum Thain, Tolga Bolukbasi

(Bolukbasi et al., 2016) demonstrated that pretrained word embeddings can inherit gender bias from the data they were trained on. We investigate how this bias affects downstream cl…

stat.ML20198 cited

Limitations of Pinned AUC for Measuring Unintended Bias

Daniel Borkan, Lucas Dixon, John Li +3

This report examines the Pinned AUC metric introduced and highlights some of its limitations. Pinned AUC provides a threshold-agnostic measure of unintended bias in a classificatio…

cs.LG20194 cited

Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

Daniel Borkan, Lucas Dixon, Jeffrey Sorensen +2

Unintended bias in Machine Learning can manifest as systemic differences in performance for different demographic groups, potentially compounding existing challenges to fairness in…