2 citations · 8 across the 8 of their papers we have counts for
4 papers · 1 filter
ASSERT: A Measurement Pipeline for GenAI Audits
Riccardo Fogliato, Abhinav Palia, Xiawei Wang +11
Audits of generative AI (GenAI) systems often summarize behavior as a reported rate: how often the audited system complies with policy. Researchers and stakeholders use that rate t…
AI-Assisted Systematization for Evaluating GenAI Systems
Dhruv Agarwal, Emily Sheng, Chad Atalla +6
Evaluating generative AI (GenAI) systems is challenging because many targets of evaluation are broad, contested concepts, such as "reasoning," "fairness," or "creativity." When the…
Taxonomizing Representational Harms using Speech Act Theory
Emily Corvi, Hannah Washington, Stefanie Reed +9
Representational harms are widely recognized among fairness-related harms caused by generative language systems. However, their definitions are commonly under-specified. We make a…
A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications
Ahmed Magooda, Alec Helyar, Kyle Jackson +14
We present a framework for the automated measurement of responsible AI (RAI) metrics for large language models (LLMs) and associated products and services. Our framework for automa…