5 papers
Data Annotation as Measurement
Emma Harvey, Allison Koenecke, Rene F. Kizilcec
Modern AI systems depend on annotated data, but annotation is rarely treated as the act of measurement that it is. Instead, annotation quality is commonly reduced to agreement: if…
Fairness-in-the-Workflow: How Machine Learning Practitioners at Big Tech Companies Approach Fairness in Recommender Systems
Jing Nathan Yan, Emma Harvey, Junxiong Wang +2
Recommender systems (RS), which are widely deployed across high-stakes domains, are susceptible to biases that can cause large-scale societal impacts. Researchers have proposed met…
Free Independence is not definable
William Boulanger, Jakub Curda, Emma Harvey +2
Free independence is an important tool for studying the structure of operator algebras. It is natural to ask from the model-theoretic standpoint whether free independence is captur…
A Framework for Auditing Chatbots for Dialect-Based Quality-of-Service Harms
Emma Harvey, Rene F. Kizilcec, Allison Koenecke
Increasingly, individuals who engage in online activities are expected to interact with large language model (LLM)-based chatbots. Prior work has shown that LLMs can display dialec…
"Don't Forget the Teachers": Towards an Educator-Centered Understanding of Harms from Large Language Models in Education
Emma Harvey, Allison Koenecke, Rene F. Kizilcec
Education technologies (edtech) are increasingly incorporating new features built on large language models (LLMs), with the goals of enriching the processes of teaching and learnin…