16 citations · 23 across the 9 of their papers we have counts for
4 papers · 1 filter
Incorporating Worker Perspectives into MTurk Annotation Practices for NLP
Olivia Huang, Eve Fleisig, Dan Klein
Current practices regarding data collection for natural language processing on Amazon Mechanical Turk (MTurk) often rely on a combination of studies on data quality and heuristics…
Ghostbuster: Detecting Text Ghostwritten by Large Language Models
Vivek Verma, Eve Fleisig, Nicholas Tomlin +1
We introduce Ghostbuster, a state-of-the-art system for detecting AI-generated text. Our method works by passing documents through a series of weaker language models, running a str…
Centering the Margins: Outlier-Based Identification of Harmed Populations in Toxicity Detection
Vyoma Raman, Eve Fleisig, Dan Klein
The impact of AI models on marginalized communities has traditionally been measured by identifying performance differences between specified demographic subgroups. Though this appr…
When the Majority is Wrong: Modeling Annotator Disagreement for Subjective Tasks
Eve Fleisig, Rediet Abebe, Dan Klein
Though majority vote among annotators is typically used for ground truth labels in natural language processing, annotator disagreement in tasks such as hate speech detection may re…