Pitfalls of Explainable ML: An Industry Perspective
arXiv:2106.07758
Abstract
As machine learning (ML) systems take a more prominent and central role in contributing to life-impacting decisions, ensuring their trustworthiness and accountability is of utmost importance. Explanations sit at the core of these desirable attributes of a ML system. The emerging field is frequently called ``Explainable AI (XAI)'' or ``Explainable ML.'' The goal of explainable ML is to intuitively explain the predictions of a ML system, while adhering to the needs to various stakeholders. Many explanation techniques were developed with contributions from both academia and industry. However, there are several existing challenges that have not garnered enough interest and serve as roadblocks to widespread adoption of explainable ML. In this short paper, we enumerate challenges in explainable ML from an industry perspective. We hope these challenges will serve as promising future research directions, and would contribute to democratizing explainable ML.
Presented at JOURNE workshop at MLSYS 2021 (https://sites.google.com/view/workshop-journe/home)
References in corpus (5)
- Towards A Rigorous Science of Interpretable Machine Learning
- Explainable AI for Trees: From Local Explanations to Global Understanding
- How can I choose an explainer? An Application-grounded Evaluation of Post-hoc Explanations
- Monitoring and explainability of models in production
- On quantitative aspects of model interpretability