Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloud

Abstract

Understanding the predictions made by machine learning (ML) models and their potential biases remains a challenging and labor-intensive task that depends on the application, the dataset, and the specific model. We present Amazon SageMaker Clarify, an explainability feature for Amazon SageMaker that launched in December 2020, providing insights into data and ML models by identifying biases and explaining predictions. It is deeply integrated into Amazon SageMaker, a fully managed service that enables data scientists and developers to build, train, and deploy ML models at any scale. Clarify supports bias detection and feature importance computation across the ML lifecycle, during data preparation, model evaluation, and post-deployment monitoring. We outline the desiderata derived from customer input, the modular architecture, and the methodology for bias and explanation computations. Further, we describe the technical challenges encountered and the tradeoffs we had to make. For illustration, we discuss two customer use cases. We present our deployment results including qualitative customer feedback and a quantitative evaluation. Finally, we summarize lessons learned, and discuss best practices for the successful adoption of fairness and explanation tools in practice.

References in corpus (18)

• A Unified Approach to Interpreting Model Predictions
• Axiomatic Attribution for Deep Networks
• European Union regulations on algorithmic decision-making and a "right to explanation"
• Model Cards for Model Reporting
• Equality of Opportunity in Supervised Learning
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• Counterfactual Fairness
• Improving fairness in machine learning systems: What do industry practitioners need?
• Fairness of Exposure in Rankings
• The What-If Tool: Interactive Probing of Machine Learning Models
• Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search
• The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons
• Avoiding Discrimination through Causal Reasoning
• Feature relevance quantification in explainable AI: A causal problem
• Empirical Risk Minimization under Fairness Constraints
• FlipTest: Fairness Testing via Optimal Transport
• LiFT: A Scalable Framework for Measuring Fairness in ML Applications
• Defuse: Harnessing Unrestricted Adversarial Examples for Debugging Models Beyond Test Accuracy

Cited by in corpus (7)

• A Critical Survey on Fairness Benefits of Explainable AI
• Inherent Limitations of AI Fairness
• A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations
• Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools
• Machine Learning practices and infrastructures
• Making the Right Thing: Bridging HCI and Responsible AI in Early-Stage AI Concept Selection
• Evaluation of AI Ethics Tools in Language Models: A Developers' Perspective Case Study