2.1k citations · 3k across the 25 of their papers we have counts for
14 papers · 1 filter
Bridging Predictions and Interventions: An Integrated Framework for Automated Decision-Systems
Inioluwa Deborah Raji, Lydia T. Liu, Angela Zhou +27
Automated decision systems (ADS) leverage predictions about individual future outcomes to inform consequential decision-making in organizational settings. Across various settings -…
The California Report on Frontier AI Policy
Rishi Bommasani, Scott R. Singer, Ruth E. Appel +20
The innovations emerging at the frontier of artificial intelligence (AI) are poised to create historic opportunities for humanity but also raise complex policy challenges. Continue…
Aggregated Individual Reporting for Post-Deployment Evaluation
Jessica Dai, Inioluwa Deborah Raji, Benjamin Recht +1
The need for developing model evaluations beyond static benchmarking, especially in the post-deployment phase, is now well-understood. At the same time, concerns about the concentr…
Evaluating Prediction-based Interventions with Human Decision Makers In Mind
Inioluwa Deborah Raji, Lydia Liu
Automated decision systems (ADS) are broadly deployed to inform and support human decision-making across a wide range of consequential settings. However, various context-specific d…
From Individual Experience to Collective Evidence: A Reporting-Based Framework for Identifying Systemic Harms
Jessica Dai, Paula Gradu, Inioluwa Deborah Raji +1
When an individual reports a negative interaction with some system, how can their personal experience be contextualized within broader patterns of system behavior? We study the rep…
Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling
Victor Ojewale, Ryan Steed, Briana Vecchione +2
Audits are critical mechanisms for identifying the risks and limitations of deployed artificial intelligence (AI) systems. However, the effective execution of AI audits remains inc…
AI auditing: The Broken Bus on the Road to AI Accountability
Abeba Birhane, Ryan Steed, Victor Ojewale +2
One of the most concrete measures to take towards meaningful AI accountability is to consequentially assess and report the systems' performance and impact. However, the practical n…
Concrete Problems in AI Safety, Revisited
Inioluwa Deborah Raji, Roel Dobbe
As AI systems proliferate in society, the AI community is increasingly preoccupied with the concept of AI Safety, namely the prevention of failures due to accidents that arise from…
Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem
Sasha Costanza-Chock, Emma Harvey, Inioluwa Deborah Raji +2
AI audits are an increasingly popular mechanism for algorithmic accountability; however, they remain poorly defined. Without a clear understanding of audit practices, let alone wid…
Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance
Inioluwa Deborah Raji, Peggy Xu, Colleen Honigsberg +1
Much attention has focused on algorithmic audits and impact assessments to hold developers and users of algorithmic systems accountable. But existing algorithmic accountability pol…
ABOUT ML: Annotation and Benchmarking on Understanding and Transparency of Machine Learning Lifecycles
Inioluwa Deborah Raji, Jingying Yang
We present the "Annotation and Benchmarking on Understanding and Transparency of Machine Learning Lifecycles" (ABOUT ML) project as an initiative to operationalize ML transparency…
Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing
Inioluwa Deborah Raji, Andrew Smart, Rebecca N. White +6
Rising concern for the societal implications of artificial intelligence systems has inspired a wave of academic and journalistic literature in which deployed systems are audited fo…
Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing
Inioluwa Deborah Raji, Timnit Gebru, Margaret Mitchell +3
Although essential to revealing biased performance, well intentioned attempts at algorithmic auditing can have effects that may harm the very populations these measures are meant t…
On the Legal Compatibility of Fairness Definitions
Alice Xiang, Inioluwa Deborah Raji
Past literature has been effective in demonstrating ideological gaps in machine learning (ML) fairness definitions when considering their use in complex socio-technical systems. Ho…