CERTIFAI: Counterfactual Explanations for Robustness, Transparency, Interpretability, and Fairness of Artificial Intelligence models

Abstract

As artificial intelligence plays an increasingly important role in our society, there are ethical and moral obligations for both businesses and researchers to ensure that their machine learning models are designed, deployed, and maintained responsibly. These models need to be rigorously audited for fairness, robustness, transparency, and interpretability. A variety of methods have been developed that focus on these issues in isolation, however, managing these methods in conjunction with model development can be cumbersome and timeconsuming. In this paper, we introduce a unified and model-agnostic approach to address these issues: Counterfactual Explanations for Robustness, Transparency, Interpretability, and Fairness of Artificial Intelligence models (CERTIFAI). Unlike previous methods in this domain, CERTIFAI is a general tool that can be applied to any black-box model and any type of input data. Given a model and an input instance, CERTIFAI uses a custom genetic algorithm to generate counterfactuals: instances close to the input that change the prediction of the model. We demonstrate how these counterfactuals can be used to examine issues of robustness, interpretability, transparency, and fairness. Additionally, we introduce CERScore, the first black-box model robustness score that performs comparably to methods that have access to model internals.

References in corpus (2)

• XGBoost: A Scalable Tree Boosting System
• Actionable Recourse in Linear Classification

Cited by in corpus (20)

• Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations
• A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability
• Towards Unifying Feature Attribution and Counterfactual Explanations: Different Means to the Same End
• Counterfactual Shapley Additive Explanations
• Fairness Score and Process Standardization: Framework for Fairness Certification in Artificial Intelligence Systems
• A Critical Survey on Fairness Benefits of Explainable AI
• A Framework and Benchmarking Study for Counterfactual Generating Methods on Tabular Data
• Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools
• Rethinking Certification for Trustworthy Machine Learning-Based Applications
• Evolutionary approaches to explainable machine learning
• Achieving Diversity in Counterfactual Explanations: a Review and Discussion
• Feature-based Learning for Diverse and Privacy-Preserving Counterfactual Explanations
• On the Connection between Game-Theoretic Feature Attributions and Counterfactual Explanations
• The Intriguing Relation Between Counterfactual Explanations and Adversarial Examples
• The Importance of Time in Causal Algorithmic Recourse
• For Better or Worse: The Impact of Counterfactual Explanations' Directionality on User Behavior in xAI
• Framing Algorithmic Recourse for Anomaly Detection
• Flexible and Robust Counterfactual Explanations with Minimal Satisfiable Perturbations
• Algorithmic Fairness Datasets: the Story so Far
• Navigating Time's Possibilities: Plausible Counterfactual Explanations for Multivariate Time-Series Forecast through Genetic Algorithms