How to Assess Trustworthy AI in Practice
arXiv:2206.09887 · doi:10.1007/s44206-023-00063-1
Abstract
This report is a methodological reflection on Z-Inspection. Z-Inspection is a holistic process used to evaluate the trustworthiness of AI-based technologies at different stages of the AI lifecycle. It focuses, in particular, on the identification and discussion of ethical issues and tensions through the elaboration of socio-technical scenarios. It uses the general European Union's High-Level Expert Group's (EU HLEG) guidelines for trustworthy AI. This report illustrates for both AI researchers and AI practitioners how the EU HLEG guidelines for trustworthy AI can be applied in practice. We share the lessons learned from conducting a series of independent assessments to evaluate the trustworthiness of AI systems in healthcare. We also share key recommendations and practical suggestions on how to ensure a rigorous trustworthy AI assessment throughout the life-cycle of an AI system.
arXiv commentOn behalf of the Z-Inspection initiative (2022)
References in corpus (11)
- Artificial Intelligence: the global landscape of ethics guidelines
- The Ethics of AI Ethics -- An Evaluation of Guidelines
- Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations
- Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem
- BS-Net: learning COVID-19 pneumonia severity on a large Chest X-Ray dataset
- Conformity Assessments and Post-market Monitoring: A Guide to the Role of Auditing in the Proposed European AI Regulation
- Linking Artificial Intelligence Principles
- On Interpretability of Deep Learning based Skin Lesion Classifiers using Concept Activation Vectors
- Axes for Sociotechnical Inquiry in AI Research
- Reward Reports for Reinforcement Learning
- Using Sentence Embeddings and Semantic Similarity for Seeking Consensus when Assessing Trustworthy AI