Adversarial Infidelity Learning for Model Interpretation

Abstract

Model interpretation is essential in data mining and knowledge discovery. It can help understand the intrinsic model working mechanism and check if the model has undesired characteristics. A popular way of performing model interpretation is Instance-wise Feature Selection (IFS), which provides an importance score of each feature representing the data samples to explain how the model generates the specific output. In this paper, we propose a Model-agnostic Effective Efficient Direct (MEED) IFS framework for model interpretation, mitigating concerns about sanity, combinatorial shortcuts, model identifiability, and information transmission. Also, we focus on the following setting: using selected features to directly predict the output of the given model, which serves as a primary evaluation metric for model-interpretation methods. Apart from the features, we involve the output of the given model as an additional input to learn an explainer based on more accurate information. To learn the explainer, besides fidelity, we propose an Adversarial Infidelity Learning (AIL) mechanism to boost the explanation learning by screening relatively unimportant features. Through theoretical and experimental analysis, we show that our AIL mechanism can help learn the desired conditional distribution between selected features and targets. Moreover, we extend our framework by integrating efficient interpretation methods as proper priors to provide a warm start. Comprehensive empirical evaluation results are provided by quantitative metrics and human evaluation to demonstrate the effectiveness and superiority of our proposed method. Our code is publicly available online at https://github.com/langlrsw/MEED.

26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '20), August 23--27, 2020, Virtual Event, USA

References in corpus (22)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• TensorFlow: A system for large-scale machine learning
• A Unified Approach to Interpreting Model Predictions
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Axiomatic Attribution for Deep Networks
• Striving for Simplicity: The All Convolutional Net
• Learning Important Features Through Propagating Activation Differences
• SmoothGrad: removing noise by adding noise
• Sanity Checks for Saliency Maps
• The relativistic discriminator: a key element missing from standard GAN
• Attention is not Explanation
• A Benchmark for Interpretability Methods in Deep Neural Networks
• Explanations can be manipulated and geometry is to blame
• Model Agnostic Supervised Local Explanations
• Fooling Neural Network Interpretations via Adversarial Model Manipulation
• CXPlain: Causal Explanations for Model Interpretation under Uncertainty
• On the (In)fidelity and Sensitivity for Explanations
• Explicability? Legibility? Predictability? Transparency? Privacy? Security? The Emerging Landscape of Interpretable Agent Behavior
• A Game Theoretic Approach to Class-wise Selective Rationalization
• Invariant Rationalization
• Interpretable Deep Learning under Fire
• Improving Feature Attribution through Input-specific Network Pruning

Cited by in corpus (5)

• From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI
• Reinforced Causal Explainer for Graph Neural Networks
• Towards interpreting ML-based automated malware detection models: a survey
• Machine Learning for Multimodal Electronic Health Records-based Research: Challenges and Perspectives
• Why Attentions May Not Be Interpretable?