Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations

Abstract

As post hoc explanation methods are increasingly being leveraged to explain complex models in high-stakes settings, it becomes critical to ensure that the quality of the resulting explanations is consistently high across various population subgroups including the minority groups. For instance, it should not be the case that explanations associated with instances belonging to a particular gender subgroup (e.g., female) are less accurate than those associated with other genders. However, there is little to no research that assesses if there exist such group-based disparities in the quality of the explanations output by state-of-the-art explanation methods. In this work, we address the aforementioned gaps by initiating the study of identifying group-based disparities in explanation quality. To this end, we first outline the key properties which constitute explanation quality and where disparities can be particularly problematic. We then leverage these properties to propose a novel evaluation framework which can quantitatively measure disparities in the quality of explanations output by state-of-the-art methods. Using this framework, we carry out a rigorous empirical analysis to understand if and when group-based disparities in explanation quality arise. Our results indicate that such disparities are more likely to occur when the models being explained are complex and highly non-linear. In addition, we also observe that certain post hoc explanation methods (e.g., Integrated Gradients, SHAP) are more likely to exhibit the aforementioned disparities. To the best of our knowledge, this work is the first to highlight and study the problem of group-based disparities in explanation quality. In doing so, our work sheds light on previously unexplored ways in which explanation methods may introduce unfairness in real world decision making.

As presented at AIES 2022

References in corpus (15)

• Towards A Rigorous Science of Interpretable Machine Learning
• Captum: A unified and generic model interpretability library for PyTorch
• Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment
• Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs
• Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods
• Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs
• How can I choose an explainer? An Application-grounded Evaluation of Post-hoc Explanations
• "Why Should You Trust My Explanation?" Understanding Uncertainty in LIME Explanations
• Machine Learning Explainability for External Stakeholders
• "How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations
• Robustness Disparities in Commercial Face Detection
• Can Explainable AI Explain Unfairness? A Framework for Evaluating Explainable AI
• Global Explanations of Neural Networks: Mapping the Landscape of Predictions
• Synthetic Benchmarks for Scientific Research in Explainable Machine Learning
• Classification by Set Cover: The Prototype Vector Machine