Ethics-Based Auditing of Automated Decision-Making Systems: Intervention Points and Policy Implications

Abstract

Organisations increasingly use automated decision-making systems (ADMS) to inform decisions that affect humans and their environment. While the use of ADMS can improve the accuracy and efficiency of decision-making processes, it is also coupled with ethical challenges. Unfortunately, the governance mechanisms currently used to oversee human decision-making often fail when applied to ADMS. In previous work, we proposed that ethics-based auditing (EBA), i.e. a structured process by which ADMS are assessed for consistency with relevant principles or norms, can (a) help organisations verify claims about their ADMS and (b) provide decision-subjects with justifications for the outputs produced by ADMS. In this article, we outline the conditions under which EBA procedures can be feasible and effective in practice. First, we argue that EBA is best understood as a 'soft' yet 'formal' governance mechanism. This implies that the main responsibility of auditors should be to spark ethical deliberation at key intervention points throughout the software development process and ensure that there is sufficient documentation to respond to potential inquiries. Second, we frame ADMS as parts of larger socio-technical systems to demonstrate that to be feasible and effective, EBA procedures must link to intervention points that span all levels of organisational governance and all phases of the software lifecycle. The main function of EBA should therefore be to inform, formalise, assess, and interlink existing governance structures. Finally, we discuss the policy implications of our findings. To support the emergence of feasible and effective EBA procedures, policymakers and regulators could provide standardised reporting formats, facilitate knowledge exchange, provide guidance on how to resolve normative tensions, and create an independent body to oversee EBA of ADMS.

arXiv commentArtificial Intelligence, Auditing, Automated Decision-Making, Ethics, Governance, Policy, Sociotechnical systems

References in corpus (23)

• Model Cards for Model Reporting
• The Ethics of AI Ethics -- An Evaluation of Guidelines
• Counterfactual Fairness
• Artificial Intelligence, Values and Alignment
• Society-in-the-Loop: Programming the Algorithmic Social Contract
• Formal Specification and Verification of Autonomous Robotic Systems: A Survey
• Ethics-Based Auditing to Develop Trustworthy AI
• Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
• Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations
• Local Rule-Based Explanations of Black Box Decision Systems
• Conformity Assessments and Post-market Monitoring: A Guide to the Role of Auditing in the Proposed European AI Regulation
• The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards
• Does Object Recognition Work for Everyone?
• Understanding artificial intelligence ethics and safety
• Aequitas: A Bias and Fairness Audit Toolkit
• Challenges of Scaled Agile for Safety-Critical Systems
• Can You Trust This Prediction? Auditing Pointwise Reliability After Learning
• Open the Black Box Data-Driven Explanation of Black Box Decision Systems
• Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits
• Axes for Sociotechnical Inquiry in AI Research
• FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning
• Measuring the Importance of User-Generated Content to Search Engines
• Hard Choices and Hard Limits for Artificial Intelligence

Cited by in corpus (5)

• Auditing large language models: a three-layered approach
• Conformity Assessments and Post-market Monitoring: A Guide to the Role of Auditing in the Proposed European AI Regulation
• Operationalising AI governance through ethics-based auditing: An industry case study
• Challenges and Best Practices in Corporate AI Governance:Lessons from the Biopharmaceutical Industry
• The Switch, the Ladder, and the Matrix: Models for Classifying AI Systems