activity
20192022
most citedLimitations of Pinned AUC for Measuring Unintended Bias

8 citations · 20 across the 3 of their papers we have counts for

collaborators

7 papers

cs.CL20228 cited

A New Generation of Perspective API: Efficient Multilingual Character-level Transformers

Alyssa Lees, Vinh Q. Tran, Yi Tay +4

On the world wide web, toxic content detectors are a crucial line of defense against potentially hateful and offensive messages. As such, building highly effective classifiers that…

cs.CL2021

Civil Rephrases Of Toxic Texts With Self-Supervised Transformers

Leo Laugier, John Pavlopoulos, Jeffrey Sorensen +1

Platforms that support online commentary, from social networks to news sites, are increasingly leveraging machine learning to assist their moderation efforts. But this process does…

cs.CL2020

Six Attributes of Unhealthy Conversation

Ilan Price, Jordan Gifford-Moore, Jory Fleming +6

We present a new dataset of approximately 44000 comments labeled by crowdworkers. Each comment is labelled as either 'healthy' or 'unhealthy', in addition to binary labels for the…

cs.CL2020

Toxicity Detection: Does Context Really Matter?

John Pavlopoulos, Jeffrey Sorensen, Lucas Dixon +2

Moderation is crucial to promoting healthy on-line discussions. Although several `toxicity' detection datasets and models have been published, most of them ignore the context of th…

cs.CL2020

Classifying Constructive Comments

Varada Kolhatkar, Nithum Thain, Jeffrey Sorensen +2

We introduce the Constructive Comments Corpus (C3), comprised of 12,000 annotated news comments, intended to help build new tools for online communities to improve the quality of t…

stat.ML20198 cited

Limitations of Pinned AUC for Measuring Unintended Bias

Daniel Borkan, Lucas Dixon, John Li +3

This report examines the Pinned AUC metric introduced and highlights some of its limitations. Pinned AUC provides a threshold-agnostic measure of unintended bias in a classificatio…