Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims

Abstract

With the recent wave of progress in artificial intelligence (AI) has come a growing awareness of the large-scale impacts of AI systems, and recognition that existing regulations and norms in industry and academia are insufficient to ensure responsible AI development. In order for AI developers to earn trust from system users, customers, civil society, governments, and other stakeholders that they are building AI responsibly, they will need to make verifiable claims to which they can be held accountable. Those outside of a given organization also need effective means of scrutinizing such claims. This report suggests various steps that different stakeholders can take to improve the verifiability of claims made about AI systems and their associated development processes, with a focus on providing evidence about the safety, security, fairness, and privacy protection of AI systems. We analyze ten mechanisms for this purpose--spanning institutions, software, and hardware--and make recommendations aimed at implementing, exploring, or improving those mechanisms.

Cited by in corpus (33)

• A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
• Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems
• Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practices
• Ethics-Based Auditing to Develop Trustworthy AI
• Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem
• "Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction
• Actionable Principles for Artificial Intelligence Policy: Three Pathways
• Harms from Increasingly Agentic Algorithmic Systems
• Trustworthy Graph Neural Networks: Aspects, Methods and Trends
• A Comparison of Natural Language Understanding Platforms for Chatbots in Software Engineering
• Logic Explained Networks
• Black-Box Access is Insufficient for Rigorous AI Audits
• Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions
• Ethics-Based Auditing of Automated Decision-Making Systems: Intervention Points and Policy Implications
• Mediating Artificial Intelligence Developments through Negative and Positive Incentives
• Filling gaps in trustworthy development of AI
• Certification Labels for Trustworthy AI: Insights From an Empirical Mixed-Method Study
• Towards Training Reproducible Deep Learning Models
• Ethical behavior in humans and machines -- Evaluating training data quality for beneficial machine learning
• Towards a Robust and Trustworthy Machine Learning System Development: An Engineering Perspective
• AI Challenges for Society and Ethics
• Automation and AI Technology in Surface Mining With a Brief Introduction to Open-Pit Operations in the Pilbara
• CLEVR-X: A Visual Reasoning Dataset for Natural Language Explanations
• Axes for Sociotechnical Inquiry in AI Research
• Yes We Care! -- Certification for Machine Learning Methods through the Care Label Framework
• Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systems
• Foundations for the Future: Institution building for the purpose of Artificial Intelligence governance
• Advancing Trustworthy AI for Sustainable Development: Recommendations for Standardising AI Incident Reporting
• A Dataset of Coordinated Cryptocurrency-Related Social Media Campaigns
• PyTorch, Explain! A Python library for Logic Explained Networks
• Trustworthy AI Inference Systems: An Industry Research View
• A Decentralized Approach towards Responsible AI in Social Ecosystems
• Outlining Traceability: A Principle for Operationalizing Accountability in Computing Systems