Publications (9)
Every Eval Ever: A Unifying Schema and Community Repository for AI Evaluation Results
Jan Batzner, Sree Harsha Nelaturu, Damian Stachura +45
AI evaluations are widely used for testing and understanding progress. However, the diverse evaluators bring with them inconsistencies that challenge analysis and comparison. First…
How Viable are Energy Savings in Smart Homes? A Call to Embrace Rebound Effects in Sustainable HCI
Christina Bremer, Harshit Gujral, Michelle Lin +3
As part of global climate action, digital technologies are seen as a key enabler of energy efficiency savings. A popular application domain for this work is smart homes. There is a…
Alberta Wells Dataset: Pinpointing Oil and Gas Wells from Satellite Imagery
Pratinav Seth, Michelle Lin, Brefo Dwamena Yaw +3
Millions of abandoned oil and gas wells are scattered across the world, leaching methane into the atmosphere and toxic compounds into the groundwater. Many of these locations are u…
Data-Centric Green AI: An Exploratory Empirical Study
Roberto Verdecchia, LuÃs Cruz, June Sallou +3
With the growing availability of large-scale datasets, and the popularization of affordable storage and computational capabilities, the energy consumed by AI is becoming a growing…
Evaluation Cards: An Interpretive Layer for AI Evaluation Reporting
Avijit Ghosh, Anka Reuel, Jenny Chim +45
AI evaluation results are produced at scale but reported inconsistently across leaderboards, model cards, benchmark papers, and company blogs. The cost is interpretive: readers can…
Challenges to Grassroots Organization Engagement with AI Policy
Carter Buckner, Jennifer Mickel, Nandhini Swaminathan +6
Public policies are being developed around the world to address privacy, economic, intellectual property, energy, and other risks that AI technologies pose. Involvement from the ge…
Bias Analysis in Unconditional Image Generative Models
Xiaofeng Zhang, Michelle Lin, Simon Lacoste-Julien +2
The widespread adoption of generative AI models has raised growing concerns about representational harm and potential discriminatory outcomes. Yet, despite growing literature on th…
Harms from Increasingly Agentic Algorithmic Systems
Alan Chan, Rebecca Salganik, Alva Markelius +19
Research in Fairness, Accountability, Transparency, and Ethics (FATE) has established many sources and forms of algorithmic harm, in domains as diverse as health care, finance, pol…
Evaluating the Social Impact of Generative AI Systems in Systems and Society
Irene Solaiman, Zeerak Talat, William Agnew +28
Generative AI systems across modalities, ranging from text (including code), image, audio, and video, have broad social impacts, but there is no official standard for means of eval…