4 citations · 5 across the 4 of their papers we have counts for
4 papers
"I want to be pushed, I want to grow": Enabling social workers to design evaluations of LLM augmentation in their work
Anna Kawakami, Chloe Qianhui Zhao, Renee Shelby +3
Workers are increasingly asked to adopt AI systems to assist their work, yet are rarely given a voice in defining what meaningful AI augmentation should look like or how to evaluat…
Quantifying the Statistical Effect of Rubric Modifications on Human-Autorater Agreement
Jessica Huynh, Alfredo Gomez, Athiya Deviyani +3
Autoraters, also referred to as LLM-as-judges, are increasingly used for evaluation and automated content moderation. However, there is limited statistical analysis of how modifica…
Offline Evaluation of Set-Based Text-to-Image Generation
Negar Arabzadeh, Fernando Diaz, Junfeng He
Text-to-Image (TTI) systems often support people during ideation, the early stages of a creative process when exposure to a broad set of relevant images can help explore the design…
Pessimistic Evaluation
Fernando Diaz
Traditional evaluation of information access systems has focused primarily on average utility across a set of information needs (information retrieval) or users (recommender system…