3 papers
cs.CY2026
Data Annotation as Measurement
Emma Harvey, Allison Koenecke, Rene F. Kizilcec
Modern AI systems depend on annotated data, but annotation is rarely treated as the act of measurement that it is. Instead, annotation quality is commonly reduced to agreement: if…
cs.HC2026
Fairness-in-the-Workflow: How Machine Learning Practitioners at Big Tech Companies Approach Fairness in Recommender Systems
Jing Nathan Yan, Emma Harvey, Junxiong Wang +2
Recommender systems (RS), which are widely deployed across high-stakes domains, are susceptible to biases that can cause large-scale societal impacts. Researchers have proposed met…
cs.CY2025
A Framework for Auditing Chatbots for Dialect-Based Quality-of-Service Harms
Emma Harvey, Rene F. Kizilcec, Allison Koenecke
Increasingly, individuals who engage in online activities are expected to interact with large language model (LLM)-based chatbots. Prior work has shown that LLMs can display dialec…