How to Assess Trustworthy AI in Practice

Abstract

This report is a methodological reflection on Z-Inspection${}^{\circledR }$. Z-Inspection${}^{\circledR }$ is a holistic process used to evaluate the trustworthiness of AI-based technologies at different stages of the AI lifecycle. It focuses, in particular, on the identification and discussion of ethical issues and tensions through the elaboration of socio-technical scenarios. It uses the general European Union's High-Level Expert Group's (EU HLEG) guidelines for trustworthy AI. This report illustrates for both AI researchers and AI practitioners how the EU HLEG guidelines for trustworthy AI can be applied in practice. We share the lessons learned from conducting a series of independent assessments to evaluate the trustworthiness of AI systems in healthcare. We also share key recommendations and practical suggestions on how to ensure a rigorous trustworthy AI assessment throughout the life-cycle of an AI system.

On behalf of the Z-Inspection${}^{\circledR }$ initiative (2022)

References in corpus (6)

• Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations
• Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem
• Conformity Assessments and Post-market Monitoring: A Guide to the Role of Auditing in the Proposed European AI Regulation
• Linking Artificial Intelligence Principles
• Reward Reports for Reinforcement Learning
• Using Sentence Embeddings and Semantic Similarity for Seeking Consensus when Assessing Trustworthy AI

Cited by in corpus (2)

• Auditing of AI: Legal, Ethical and Technical Approaches
• Assessing the Applicability of Existing Design Recommendations to AI Companion Design: A Multi-Method Study