Towards A Rigorous Science of Interpretable Machine Learning

Abstract

As machine learning systems become ubiquitous, there has been a surge of interest in interpretable machine learning: systems that provide explanation for their outputs. These explanations are often used to qualitatively assess other criteria such as safety or non-discrimination. However, despite the interest in interpretability, there is very little consensus on what interpretable machine learning is and how it should be measured. In this position paper, we first define interpretability and describe when interpretability is needed (and when it is not). Next, we suggest a taxonomy for rigorous evaluation and expose open questions towards a more rigorous science of interpretable machine learning.

References in corpus (5)

• Playing Atari with Deep Reinforcement Learning
• European Union regulations on algorithmic decision-making and a "right to explanation"
• OpenML: networked science in machine learning
• "Why Should I Trust You?": Explaining the Predictions of Any Classifier
• Meta-Unsupervised-Learning: A supervised approach to unsupervised learning

Cited by in corpus (544)

• A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
• On the Opportunities and Risks of Foundation Models
• Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
• Explainable artificial intelligence (XAI) in deep learning-based medical image analysis
• Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations
• Explainable Machine Learning for Scientific Insights and Discoveries
• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• A Survey on the Explainability of Supervised Machine Learning
• The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies
• Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems
• SmoothGrad: removing noise by adding noise
• GNNExplainer: Generating Explanations for Graph Neural Networks
• 'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions
• Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges
• What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research
• Towards Explainable Artificial Intelligence
• Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions
• Deep Reinforcement Learning: An Overview
• The What-If Tool: Interactive Probing of Machine Learning Models
• Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
• Expanding Explainability: Towards Social Transparency in AI systems
• Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
• The Measure and Mismeasure of Fairness
• Towards Robust Interpretability with Self-Explaining Neural Networks
• Measuring the Quality of Explanations: The System Causability Scale (SCS). Comparing Human and Machine Explanations
• Optimization Problems for Machine Learning: A Survey
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
• explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning
• What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use
• Pathologies of Neural Models Make Interpretations Difficult
• Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations
• A Review of Challenges and Opportunities in Machine Learning for Health
• Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches
• Explainable Artificial Intelligence (XAI) from a user perspective- A synthesis of prior literature and problematizing avenues for future research
• Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction
• Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI
• Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciences
• Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs
• A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?
• Local Rule-Based Explanations of Black Box Decision Systems
• Guidelines and Evaluation of Clinical Explainable AI in Medical Image Analysis
• Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art
• Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods
• A Survey of Contextual Optimization Methods for Decision Making under Uncertainty
• Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond
• Interpreting Blackbox Models via Model Extraction
• Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation
• On the Explainability of Natural Language Processing Deep Models
• An Evaluation of the Human-Interpretability of Explanation
• Scalable agent alignment via reward modeling: a research direction
• Levels of explainable artificial intelligence for human-aligned conversational explanations
• Fooling Neural Network Interpretations via Adversarial Model Manipulation
• AI Safety Gridworlds
• Machine Learning Testing: Survey, Landscapes and Horizons
• WT5?! Training Text-to-Text Models to Explain their Predictions
• Deep Neural Decision Trees
• Acquisition of Chess Knowledge in AlphaZero
• How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation
• Multi-Objective Molecule Generation using Interpretable Substructures
• Making AI Intelligible: Philosophical Foundations
• Machine Unlearning: Solutions and Challenges
• Empowering Local Communities Using Artificial Intelligence
• Explaining Classifiers with Causal Concept Effect (CaCE)
• Explainable Artificial Intelligence Approaches: A Survey
• How can I choose an explainer? An Application-grounded Evaluation of Post-hoc Explanations
• Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
• Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
• Benchmarking Attribution Methods with Relative Feature Importance
• Towards Automatic Concept-based Explanations
• Fairness and Transparency in Recommendation: The Users' Perspective
• A deep inverse reinforcement learning approach to route choice modeling with context-dependent rewards
• Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems
• Evaluating Reinforcement Learning Algorithms in Observational Health Settings
• Model-Agnostic Counterfactual Explanations for Consequential Decisions
• Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI
• Explainable AI: current status and future directions
• Logic Explained Networks
• Explainable Artificial Intelligence for Process Mining: A General Overview and Application of a Novel Local Explanation Approach for Predictive Process Monitoring
• Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems
• XAIR: A Framework of Explainable AI in Augmented Reality
• Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions
• Algorithmic Recourse: from Counterfactual Explanations to Interventions
• Explainability in Music Recommender Systems
• "Why Should You Trust My Explanation?" Understanding Uncertainty in LIME Explanations
• Abstractive Text Summarization: State of the Art, Challenges, and Improvements
• Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory
• Explainable Machine Learning for Public Policy: Use Cases, Gaps, and Research Directions
• Explainable AI in Diagnosing and Anticipating Leukemia Using Transfer Learning Method
• Using Machine Learning to Calibrate Storm-Scale Probabilistic Guidance of Severe Weather Hazards in the Warn-on-Forecast System
• NBDT: Neural-Backed Decision Trees
• Explainable AI, but explainable to whom?
• Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI
• Quantifying Model Complexity via Functional Decomposition for Better Post-Hoc Interpretability
• Entropy-based Logic Explanations of Neural Networks
• CrypTen: Secure Multi-Party Computation Meets Machine Learning
• Artificial Intelligence in Glioma Imaging: Challenges and Advances
• Rationalization for Explainable NLP: A Survey
• Local and Global Explanations of Agent Behavior: Integrating Strategy Summaries with Saliency Maps
• Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations
• A Survey on Legal Question Answering Systems
• On the Forces of Driver Distraction: Explainable Predictions for the Visual Demand of In-Vehicle Touchscreen Interactions
• Interpretable and intervenable ultrasonography-based machine learning models for pediatric appendicitis
• Quantum Language Model with Entanglement Embedding for Question Answering
• A Hierarchy of Limitations in Machine Learning
• Data Science vs. Statistics: Two Cultures?
• Concise Fuzzy System Modeling Integrating Soft Subspace Clustering and Sparse Learning
• Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation
• Assessing the Local Interpretability of Machine Learning Models
• Interpretable multiclass classification by MDL-based rule lists
• Guidance on the Assurance of Machine Learning in Autonomous Systems (AMLAS)
• The Role of Cooperation in Responsible AI Development
• A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learning
• On Interpretability of Artificial Neural Networks: A Survey
• A survey of algorithmic recourse: definitions, formulations, solutions, and prospects
• Outcome-Explorer: A Causality Guided Interactive Visual Interface for Interpretable Algorithmic Decision Making
• The Promise and Peril of Human Evaluation for Model Interpretability
• Toward Accurate Interpretable Predictions of Materials Properties within Transformer Language Models
• When Explanations Lie: Why Many Modified BP Attributions Fail
• Deep Weighted Averaging Classifiers
• PGM-Explainer: Probabilistic Graphical Model Explanations for Graph Neural Networks
• Getting a CLUE: A Method for Explaining Uncertainty Estimates
• Causality Learning: A New Perspective for Interpretable Machine Learning
• Interpreting a Recurrent Neural Network's Predictions of ICU Mortality Risk
• Machine Learning Explainability for External Stakeholders
• Explainability through uncertainty: Trustworthy decision-making with neural networks
• Finite Versus Infinite Neural Networks: an Empirical Study
• MAGIX: Model Agnostic Globally Interpretable Explanations
• Using Machine Learning Safely in Automotive Software: An Assessment and Adaption of Software Process Requirements in ISO 26262
• Regularizing Black-box Models for Improved Interpretability
• Counterfactual Visual Explanations
• Evaluating Explanation Without Ground Truth in Interpretable Machine Learning
• Interpretability of a Deep Learning Model in the Application of Cardiac MRI Segmentation with an ACDC Challenge Dataset
• Tree Space Prototypes: Another Look at Making Tree Ensembles Interpretable
• Interpretations are useful: penalizing explanations to align neural networks with prior knowledge
• Artificial Intelligence and Diabetes Mellitus: An Inside Look Through the Retina
• A new interpretable unsupervised anomaly detection method based on residual explanation
• Considerations When Learning Additive Explanations for Black-Box Models
• Machine Learning Software Engineering in Practice: An Industrial Case Study
• Detecting Statistical Interactions from Neural Network Weights
• Evaluating Saliency Map Explanations for Convolutional Neural Networks: A User Study
• A Survey on Neural Network Interpretability
• The Evolution of Reinforcement Learning in Quantitative Finance: A Survey
• The Price of Interpretability
• Interpretable AI-based Large-scale 3D Pathloss Prediction Model for enabling Emerging Self-Driving Networks
• Towards falsifiable interpretability research
• On the Art and Science of Machine Learning Explanations
• Interpretable Deep Models for Cardiac Resynchronisation Therapy Response Prediction
• Techniques for Interpretable Machine Learning
• Outside the Box: Abstraction-Based Monitoring of Neural Networks
• An Accelerator for Rule Induction in Fuzzy Rough Theory
• Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena
• Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
• "How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations
• The effectiveness of feature attribution methods and its correlation with automatic evaluation scores
• The Future of Misinformation Detection: New Perspectives and Trends
• Entanglement-guided architectures of machine learning by quantum tensor network
• Informing clinical assessment by contextualizing post-hoc explanations of risk prediction models in type-2 diabetes
• Explaining Human Activity Recognition with SHAP: Validating Insights with Perturbation and Quantitative Measures
• Interpreting CNNs via Decision Trees
• MoËT: Mixture of Expert Trees and its Application to Verifiable Reinforcement Learning
• Appropriate Fairness Perceptions? On the Effectiveness of Explanations in Enabling People to Assess the Fairness of Automated Decision Systems
• Privacy Preservation in Federated Learning: An insightful survey from the GDPR Perspective
• Can counterfactual explanations of AI systems' predictions skew lay users' causal intuitions about the world? If so, can we correct for that?
• Adequate and fair explanations
• How Much Automation Does a Data Scientist Want?
• Interpretable deep learning for nuclear deformation in heavy ion collisions
• Explainability Auditing for Intelligent Systems: A Rationale for Multi-Disciplinary Perspectives
• Gaussian Process Latent Class Choice Models
• Beyond Individualized Recourse: Interpretable and Interactive Summaries of Actionable Recourses
• Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?
• What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Play
• Keep Your Friends Close and Your Counterfactuals Closer: Improved Learning From Closest Rather Than Plausible Counterfactual Explanations in an Abstract Setting
• Teaching Categories to Human Learners with Visual Explanations
• TDM: Trustworthy Decision-Making via Interpretability Enhancement
• "The Human Body is a Black Box": Supporting Clinical Decision-Making with Deep Learning
• Embedding Deep Networks into Visual Explanations
• Tensor networks for interpretable and efficient quantum-inspired machine learning
• Bridging the gap: Towards an Expanded Toolkit for AI-driven Decision-Making in the Public Sector
• Scalable Rule-Based Representation Learning for Interpretable Classification
• Let's Go to the Alien Zoo: Introducing an Experimental Framework to Study Usability of Counterfactual Explanations for Machine Learning
• Analysing Deep Reinforcement Learning Agents Trained with Domain Randomisation
• Learning Interpretable Rules for Scalable Data Representation and Classification
• Enforcing Interpretability and its Statistical Impacts: Trade-offs between Accuracy and Interpretability
• Question-Driven Design Process for Explainable AI User Experiences
• AutoScore-Imbalance: An interpretable machine learning tool for development of clinical scores with rare events data
• A Human-Grounded Evaluation of SHAP for Alert Processing
• Contrastive Explanations with Local Foil Trees
• Structural Compression of Convolutional Neural Networks
• Global Explanations of Neural Networks: Mapping the Landscape of Predictions
• Exploring Sensitivity of ICF Outputs to Design Parameters in Experiments Using Machine Learning
• Fairwashing: the risk of rationalization
• Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions
• How Well do Feature Visualizations Support Causal Understanding of CNN Activations?
• Trepan Reloaded: A Knowledge-driven Approach to Explaining Artificial Neural Networks
• Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications
• Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
• Instance-Level Explanations for Fraud Detection: A Case Study
• Machine learning effective models for quantum systems
• Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience
• Evaluation of Similarity-based Explanations
• Calibrated Explanations: with Uncertainty Information and Counterfactuals
• Shapley Decomposition of R-Squared in Machine Learning Models
• Can Requirements Engineering Support Explainable Artificial Intelligence? Towards a User-Centric Approach for Explainability Requirements
• xGEMs: Generating Examplars to Explain Black-Box Models
• Machine learning and AI research for Patient Benefit: 20 Critical Questions on Transparency, Replicability, Ethics and Effectiveness
• TIP: Typifying the Interpretability of Procedures
• A Formal Framework to Characterize Interpretability of Procedures
• Human-Centered Explainable AI (XAI): From Algorithms to User Experiences
• Unraveling the Dilemma of AI Errors: Exploring the Effectiveness of Human and Machine Explanations for Large Language Models
• Interpreting Deep Classifier by Visual Distillation of Dark Knowledge
• Foundations of Explainable Knowledge-Enabled Systems
• Constant-Time Predictive Distributions for Gaussian Processes
• Proposed Guidelines for the Responsible Use of Explainable Machine Learning
• Optimization Methods for Interpretable Differentiable Decision Trees in Reinforcement Learning
• Achieving Diversity in Counterfactual Explanations: a Review and Discussion
• Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Prediction
• On the Trustworthiness of Tree Ensemble Explainability Methods
• A multi-task deep learning model for the classification of Age-related Macular Degeneration
• ViCE: Visual Counterfactual Explanations for Machine Learning Models
• Evaluations and Methods for Explanation through Robustness Analysis
• TED: Teaching AI to Explain its Decisions
• AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models
• NODE-GAM: Neural Generalized Additive Model for Interpretable Deep Learning
• Challenges and Trends in User Trust Discourse in AI
• Image Reconstruction with Predictive Filter Flow
• On quantitative aspects of model interpretability
• Interpretable Anomaly Detection with DIFFI: Depth-based Isolation Forest Feature Importance
• Effectiveness of Tree-based Ensembles for Anomaly Discovery: Insights, Batch and Streaming Active Learning
• One Map Does Not Fit All: Evaluating Saliency Map Explanation on Multi-Modal Medical Images
• Neural Logic Reinforcement Learning
• Decisions, Counterfactual Explanations and Strategic Behavior
• Best of both worlds: local and global explanations with human-understandable concepts
• Quantitative Evaluations on Saliency Methods: An Experimental Study
• Toward a Theory of Causation for Interpreting Neural Code Models
• Explaining a black-box using Deep Variational Information Bottleneck Approach
• Causal Interpretability for Machine Learning -- Problems, Methods and Evaluation
• Explaining Explanations to Society
• Individual Explanations in Machine Learning Models: A Survey for Practitioners
• SignalNet: A Low Resolution Sinusoid Decomposition and Estimation Network
• Making the black-box brighter: interpreting machine learning algorithm for forecasting drilling accidents
• Explanation by Progressive Exaggeration
• FUTURE-AI: Guiding Principles and Consensus Recommendations for Trustworthy Artificial Intelligence in Medical Imaging
• The Impact of Explanations on Fairness in Human-AI Decision-Making: Protected vs Proxy Features
• Personalized explanation in machine learning: A conceptualization
• Interpretable Machine Learning: Moving From Mythos to Diagnostics
• Efficient Learning of Interpretable Classification Rules
• Learning Interpretable Models with Causal Guarantees
• Sequential Interpretability: Methods, Applications, and Future Direction for Understanding Deep Learning Models in the Context of Sequential Data
• Local Post-Hoc Explanations for Predictive Process Monitoring in Manufacturing
• E-Commerce Dispute Resolution Prediction
• The Importance of Being Interpretable: Toward An Understandable Machine Learning Encoder for Galaxy Cluster Cosmology
• Combating Fake News with Interpretable News Feed Algorithms
• Towards Explainable Deep Learning for Credit Lending: A Case Study
• Should We Trust (X)AI? Design Dimensions for Structured Experimental Evaluations
• What made you do this? Understanding black-box decisions with sufficient input subsets
• Question-Conditioned Counterfactual Image Generation for VQA
• Improving the Interpretability of Deep Neural Networks with Knowledge Distillation
• Modeling Heterogeneity in Mode-Switching Behavior Under a Mobility-on-Demand Transit System: An Interpretable Machine Learning Approach
• VINE: Visualizing Statistical Interactions in Black Box Models
• Hybrid Predictive Model: When an Interpretable Model Collaborates with a Black-box Model
• Disentangled Attribution Curves for Interpreting Random Forests and Boosted Trees
• Generating User-friendly Explanations for Loan Denials using GANs
• On the Semantic Interpretability of Artificial Intelligence Models
• Are Metrics Enough? Guidelines for Communicating and Visualizing Predictive Models to Subject Matter Experts
• AutoDOViz: Human-Centered Automation for Decision Optimization
• Impact of Accuracy on Model Interpretations
• Pitfalls of Explainable ML: An Industry Perspective
• RelatIF: Identifying Explanatory Training Examples via Relative Influence
• Interpretable and Pedagogical Examples
• Unexplainability and Incomprehensibility of Artificial Intelligence
• Deep Learning for Human Locomotion Analysis in Lower-Limb Exoskeletons: A Comparative Study
• Teaching Meaningful Explanations
• SLEEPER: interpretable Sleep staging via Prototypes from Expert Rules
• Traceability of Deep Neural Networks
• Multiresolution Tensor Learning for Efficient and Interpretable Spatial Analysis
• Transformation Importance with Applications to Cosmology
• Towards Explainability of Machine Learning Models in Insurance Pricing
• Do Transformer Attention Heads Provide Transparency in Abstractive Summarization?
• Stability of Internal States in Recurrent Neural Networks Trained on Regular Languages
• SUBPLEX: Towards a Better Understanding of Black Box Model Explanations at the Subpopulation Level
• Understanding Multi-Head Attention in Abstractive Summarization
• Disentangling Controllable and Uncontrollable Factors of Variation by Interacting with the World
• Interpreting Deep Neural Networks Through Variable Importance
• Rethinking Trajectory Forecasting Evaluation
• Adversarial NLI for Factual Correctness in Text Summarisation Models
• Combat COVID-19 Infodemic Using Explainable Natural Language Processing Models
• Provably efficient, succinct, and precise explanations
• XAI Methods for Neural Time Series Classification: A Brief Review
• Information Fusion via Symbolic Regression: A Tutorial in the Context of Human Health
• Does Your Model Think Like an Engineer? Explainable AI for Bearing Fault Detection with Deep Learning
• Towards a Unified Multidimensional Explainability Metric: Evaluating Trustworthiness in AI Models
• An Interaction-based Convolutional Neural Network (ICNN) Towards Better Understanding of COVID-19 X-ray Images
• A Conceptual Framework for Establishing Trust in Real World Intelligent Systems
• A Novel Interaction-based Methodology Towards Explainable AI with Better Understanding of Pneumonia Chest X-ray Images
• Algorithmic Recourse in the Wild: Understanding the Impact of Data and Model Shifts
• Micro-entries: Encouraging Deeper Evaluation of Mental Models Over Time for Interactive Data Systems
• Why an Android App is Classified as Malware? Towards Malware Classification Interpretation
• Combining Anatomical and Functional Networks for Neuropathology Identification: A Case Study on Autism Spectrum Disorder
• Deep Neural Networks for Choice Analysis: A Statistical Learning Theory Perspective
• Learning Fair Rule Lists
• Principles of Explanation in Human-AI Systems
• Discovering Fair Representations in the Data Domain
• Machine Learning Interpretability: A Science rather than a tool
• Decoding Non-Linearity and Complexity: Deep Tabular Learning Approaches for Materials Science
• Adversarial Attacks and Defenses: An Interpretation Perspective
• Debugging Tests for Model Explanations
• Plurals: A System for Guiding LLMs Via Simulated Social Ensembles
• To Trust or Not to Trust a Regressor: Estimating and Explaining Trustworthiness of Regression Predictions
• ShapeFlow: Dynamic Shape Interpreter for TensorFlow
• Concept explainability for plant diseases classification
• Order in the Court: Explainable AI Methods Prone to Disagreement
• When to Intervene: Detecting Abnormal Mood using Everyday Smartphone Conversations
• Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems
• Fairkit, Fairkit, on the Wall, Who's the Fairest of Them All? Supporting Data Scientists in Training Fair Models
• Learning Qualitatively Diverse and Interpretable Rules for Classification
• A New Approach for Explainable Multiple Organ Annotation with Few Data
• GAN-based Generation and Automatic Selection of Explanations for Neural Networks
• Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges
• Interpretable Machine Learning for COVID-19: An Empirical Study on Severity Prediction Task
• Theory-based residual neural networks: A synergy of discrete choice models and deep neural networks
• A Trustworthiness Score to Evaluate DNN Predictions
• From Heatmaps to Structural Explanations of Image Classifiers
• Explainable Agents Through Social Cues: A Review
• The Thousand Faces of Explainable AI Along the Machine Learning Life Cycle: Industrial Reality and Current State of Research
• CoDeGAN: Contrastive Disentanglement for Generative Adversarial Network
• Interpretable Model-based Hierarchical Reinforcement Learning using Inductive Logic Programming
• RecRec: Algorithmic Recourse for Recommender Systems
• Mixture of Linear Models Co-supervised by Deep Neural Networks
• Counterfactual Evaluation for Explainable AI
• Compositional Attention Networks for Interpretability in Natural Language Question Answering
• MonoNet: Towards Interpretable Models by Learning Monotonic Features
• Interpretable Learning-to-Rank with Generalized Additive Models
• Learning Credible Deep Neural Networks with Rationale Regularization
• Resisting Out-of-Distribution Data Problem in Perturbation of XAI
• "I know it when I see it". Visualization and Intuitive Interpretability
• Data Representing Ground-Truth Explanations to Evaluate XAI Methods
• Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualization
• Information Theoretic Measures for Fairness-aware Feature Selection
• Reimagining an autonomous vehicle
• High Dimensional Model Explanations: an Axiomatic Approach
• Interpretable and Personalized Apprenticeship Scheduling: Learning Interpretable Scheduling Policies from Heterogeneous User Demonstrations
• Robust and Stable Black Box Explanations
• Explainable Deep Relational Networks for Predicting Compound-Protein Affinities and Contacts
• A unified framework for evaluating the robustness of machine-learning interpretability for prospect risking
• Foundations of Symbolic Languages for Model Interpretability
• Deep Neural Networks for Choice Analysis: Extracting Complete Economic Information for Interpretation
• Play MNIST For Me! User Studies on the Effects of Post-Hoc, Example-Based Explanations & Error Rates on Debugging a Deep Learning, Black-Box Classifier
• Heteroscedastic Calibration of Uncertainty Estimators in Deep Learning
• Interactive Visualization for Debugging RL
• Independence Promoted Graph Disentangled Networks
• Harmonizing Feature Attributions Across Deep Learning Architectures: Enhancing Interpretability and Consistency
• Predicting Driver Fatigue in Automated Driving with Explainability
• NLS: an accurate and yet easy-to-interpret regression method
• Challenges for cognitive decoding using deep learning methods
• Sequential Explanations with Mental Model-Based Policies
• DeepNNK: Explaining deep models and their generalization using polytope interpolation
• Interpreting search result rankings through intent modeling
• Towards Emergent Language Symbolic Semantic Segmentation and Model Interpretability
• How Useful Are the Machine-Generated Interpretations to General Users? A Human Evaluation on Guessing the Incorrectly Predicted Labels
• State of the Art in Fair ML: From Moral Philosophy and Legislation to Fair Classifiers
• DISSECT: Disentangled Simultaneous Explanations via Concept Traversals
• Towards Human Cognition Level-based Experiment Design for Counterfactual Explanations (XAI)
• A Review of Explainable Artificial Intelligence in Manufacturing
• DIVINE: Diverse Influential Training Points for Data Visualization and Model Refinement
• Explainability in AI Based Applications: A Framework for Comparing Different Techniques
• Gradient-based Analysis of NLP Models is Manipulable
• Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?
• Knowledge-based XAI through CBR: There is more to explanations than models can tell
• Adaptive wavelet distillation from neural networks through interpretations
• Discretized Integrated Gradients for Explaining Language Models
• Teaching Responsible Data Science: Charting New Pedagogical Territory
• An Empirical Study of Explainable AI Techniques on Deep Learning Models For Time Series Tasks
• AdaCare: Explainable Clinical Health Status Representation Learning via Scale-Adaptive Feature Extraction and Recalibration
• Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships
• Generative Counterfactuals for Neural Networks via Attribute-Informed Perturbation
• A psychophysics approach for quantitative comparison of interpretable computer vision models
• Interpretable Feature Learning in Multivariate Big Data Analysis for Network Monitoring
• DuTrust: A Sentiment Analysis Dataset for Trustworthiness Evaluation
• Technologies for Trustworthy Machine Learning: A Survey in a Socio-Technical Context
• Model-agnostic interpretation by visualization of feature perturbations
• Interpretable Neural Networks based classifiers for categorical inputs
• Importance measures derived from random forests: characterisation and extension
• The Role of Individual User Differences in Interpretable and Explainable Machine Learning Systems
• Modeling Disclosive Transparency in NLP Application Descriptions
• Towards Rigorous Interpretations: a Formalisation of Feature Attribution
• Learning Predictive and Interpretable Timeseries Summaries from ICU Data
• Neural Network Complexity of Chaos and Turbulence
• Intelligence plays dice: Stochasticity is essential for machine learning
• Inducing and Embedding Senses with Scaled Gumbel Softmax
• Contextual Classification Using Self-Supervised Auxiliary Models for Deep Neural Networks
• Evaluating the performance of the LIME and Grad-CAM explanation methods on a LEGO multi-label image classification task
• Conversational AI Systems for Social Good: Opportunities and Challenges
• A Causal Lens for Peeking into Black Box Predictive Models: Predictive Model Interpretation via Causal Attribution
• Human-AI Co-Learning for Data-Driven AI
• Extracting Incentives from Black-Box Decisions
• Good-looking but Lacking Faithfulness: Understanding Local Explanation Methods through Trend-based Testing
• Convolutional Neural Network Models and Interpretability for the Anisotropic Reynolds Stress Tensor in Turbulent One-dimensional Flows
• Do ML Experts Discuss Explainability for AI Systems? A discussion case in the industry for a domain-specific solution
• EXS: Explainable Search Using Local Model Agnostic Interpretability
• Analysis Methods in Neural Language Processing: A Survey
• Towards Social Situation Awareness in Support Agents
• Clinical Predictive Models for COVID-19: Systematic Study
• Controlling Neural Networks with Rule Representations
• Towards an Explanation Space to Align Humans and Explainable-AI Teamwork
• Explainable AI Enabled Inspection of Business Process Prediction Models
• Evaluating Neural Machine Comprehension Model Robustness to Noisy Inputs and Adversarial Attacks
• State-of-the-art Techniques in Deep Edge Intelligence
• Dynamic Logistic Ensembles with Recursive Probability and Automatic Subset Splitting for Enhanced Binary Classification
• Nonlinear Semi-Parametric Models for Survival Analysis
• FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging
• Why is the prediction wrong? Towards underfitting case explanation via meta-classification
• Reasons, Values, Stakeholders: A Philosophical Framework for Explainable Artificial Intelligence
• Evaluation of importance estimators in deep learning classifiers for Computed Tomography
• On Two XAI Cultures: A Case Study of Non-technical Explanations in Deployed AI System
• A Comparative Approach to Explainable Artificial Intelligence Methods in Application to High-Dimensional Electronic Health Records: Examining the Usability of XAI
• Do RNN States Encode Abstract Phonological Processes?
• Feature Synergy, Redundancy, and Independence in Global Model Explanations using SHAP Vector Decomposition
• Consensus-based Interpretable Deep Neural Networks with Application to Mortality Prediction
• Know Your Model (KYM): Increasing Trust in AI and Machine Learning
• Investigating and Simplifying Masking-based Saliency Methods for Model Interpretability
• Understanding the Prediction Mechanism of Sentiments by XAI Visualization
• Cadre Modeling: Simultaneously Discovering Subpopulations and Predictive Models
• Training Feedforward Neural Networks with Standard Logistic Activations is Feasible
• Monitoring Trust in Human-Machine Interactions for Public Sector Applications
• Interpretability of machine-learning models in physical sciences
• Crowdsourcing Evaluation of Saliency-based XAI Methods
• Implicit Feedback Deep Collaborative Filtering Product Recommendation System
• Critical Empirical Study on Black-box Explanations in AI
• Teaching AI to Explain its Decisions Using Embeddings and Multi-Task Learning
• NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language
• A cautionary tale on fitting decision trees to data from additive models: generalization lower bounds
• Interpretable Question Answering on Knowledge Bases and Text
• A Multistakeholder Approach Towards Evaluating AI Transparency Mechanisms
• Towards Explanation of DNN-based Prediction with Guided Feature Inversion
• Label-Aware Neural Tangent Kernel: Toward Better Generalization and Local Elasticity
• SelfExplain: A Self-Explaining Architecture for Neural Text Classifiers
• Robot Mindreading and the Problem of Trust
• On Blame Attribution for Accountable Multi-Agent Sequential Decision Making
• The Mass, Fake News, and Cognition Security
• Optimal Explanations of Linear Models
• Transparent Classification with Multilayer Logical Perceptrons and Random Binarization
• On the Definition of Appropriate Trust and the Tools that Come with it
• Evaluating Explainable Methods for Predictive Process Analytics: A Functionally-Grounded Approach
• SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planning
• Many Faces of Feature Importance: Comparing Built-in and Post-hoc Feature Importance in Text Classification
• QUACKIE: A NLP Classification Task With Ground Truth Explanations
• Mathematical decisions and non-causal elements of explainable AI
• A Semiparametric Approach to Interpretable Machine Learning
• Neural Machine Translation: A Review and Survey
• Show or Suppress? Managing Input Uncertainty in Machine Learning Model Explanations
• Computability, Complexity, Consistency and Controllability: A Four C's Framework for cross-disciplinary Ethical Algorithm Research
• Leveraging traditional ecological knowledge in ecosystem restoration projects utilizing machine learning
• A Game-Based Approach for Helping Designers Learn Machine Learning Concepts
• Distilling Black-Box Travel Mode Choice Model for Behavioral Interpretation
• Why should I not follow you? Reasons For and Reasons Against in Responsible Recommender Systems
• Deep NMF Topic Modeling
• Making Document-Level Information Extraction Right for the Right Reasons
• Logic Traps in Evaluating Attribution Scores
• Interpretability is not Explainability: New Quantitative XAI Approach with a focus on Recommender Systems in Education
• The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems
• Accelerating Entrepreneurial Decision-Making Through Hybrid Intelligence
• Two4Two: Evaluating Interpretable Machine Learning - A Synthetic Dataset For Controlled Experiments
• Quality Metrics for Transparent Machine Learning With and Without Humans In the Loop Are Not Correlated
• Extracting Weighted Automata for Approximate Minimization in Language Modelling
• Provenance Graph Kernel
• Neural-to-Tree Policy Distillation with Policy Improvement Criterion
• Selecting Datasets for Evaluating an Enhanced Deep Learning Framework
• A Minimal Intervention Definition of Reverse Engineering a Neural Circuit
• The Eval4NLP Shared Task on Explainable Quality Estimation: Overview and Results
• Formalizing Interruptible Algorithms for Human over-the-loop Analytics
• Towards Interpretable ANNs: An Exact Transformation to Multi-Class Multivariate Decision Trees
• Generalized Constraints as A New Mathematical Problem in Artificial Intelligence: A Review and Perspective
• Stochastic encoding of graphs in deep learning allows for complex analysis of gender classification in resting-state and task functional brain networks from the UK Biobank
• One-shot Policy Elicitation via Semantic Reward Manipulation
• Deep Neural Networks for Choice Analysis: Architectural Design with Alternative-Specific Utility Functions
• Demystifying Brain Tumour Segmentation Networks: Interpretability and Uncertainty Analysis
• A Tour of Convolutional Networks Guided by Linear Interpreters
• Technical Report: Partial Dependence through Stratification
• Reconciling the Discrete-Continuous Divide: Towards a Mathematical Theory of Sparse Communication
• Faithful and Plausible Explanations of Medical Code Predictions
• Challenges in Vessel Behavior and Anomaly Detection: From Classical Machine Learning to Deep Learning
• Human Evaluation of Interpretability: The Case of AI-Generated Music Knowledge
• Machine Learning Interpretability and Its Impact on Smart Campus Projects
• Unifying Model Explainability and Robustness via Machine-Checkable Concepts
• Vizarel: A System to Help Better Understand RL Agents
• An Interpretable Probabilistic Approach for Demystifying Black-box Predictive Models
• Towards Evaluating Exploratory Model Building Process with AutoML Systems
• Evaluation of Local Explanation Methods for Multivariate Time Series Forecasting
• Extractive Adversarial Networks: High-Recall Explanations for Identifying Personal Attacks in Social Media Posts
• GLAD: GLocalized Anomaly Detection via Human-in-the-Loop Learning
• Explainable Neural Computation via Stack Neural Module Networks
• A Theory of Diagnostic Interpretation in Supervised Classification
• What Gets Echoed? Understanding the "Pointers" in Explanations of Persuasive Arguments
• ACE -- An Anomaly Contribution Explainer for Cyber-Security Applications
• Sampling Prediction-Matching Examples in Neural Networks: A Probabilistic Programming Approach
• Artificial Intelligence for Pediatric Ophthalmology
• Training Machine Learning Models by Regularizing their Explanations
• On the Use of Interpretable Machine Learning for the Management of Data Quality
• Differentiable Grammars for Videos
• Background Knowledge Injection for Interpretable Sequence Classification
• From Demonstrations to Task-Space Specifications: Using Causal Analysis to Extract Rule Parameterization from Demonstrations
• Games for Fairness and Interpretability
• Towards Trustworthy Deception Detection: Benchmarking Model Robustness across Domains, Modalities, and Languages
• On the Flip Side: Identifying Counterexamples in Visual Question Answering
• multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoning
• Attack-agnostic Adversarial Detection on Medical Data Using Explainable Machine Learning
• Human-Understandable Decision Making for Visual Recognition
• Measuring the Stability of Learned Features
• A Double Penalty Model for Interpretability
• Saliency strikes back: How filtering out high frequencies improves white-box explanations
• Towards Designing Computer Vision-based Explainable-AI Solution: A Use Case of Livestock Mart Industry
• Mitigating the Position Bias of Transformer Models in Passage Re-Ranking
• Visual Summary of Value-level Feature Attribution in Prediction Classes with Recurrent Neural Networks
• A Graph Neural Network Approach for Product Relationship Prediction
• "A cold, technical decision-maker": Can AI provide explainability, negotiability, and humanity?
• Explaining Documents' Relevance to Search Queries
• Iterative Rule Extension for Logic Analysis of Data: an MILP-based heuristic to derive interpretable binary classification from large datasets
• Logic Constraints to Feature Importances
• Self-explaining variational posterior distributions for Gaussian Process models
• Train, Diagnose and Fix: Interpretable Approach for Fine-grained Action Recognition
• Information-theoretic Evolution of Model Agnostic Global Explanations
• Explainable Deep Modeling of Tabular Data using TableGraphNet
• A Series of Unfortunate Counterfactual Events: the Role of Time in Counterfactual Explanations
• A general framework for scientifically inspired explanations in AI
• A Framework to Learn with Interpretation
• Understanding Information Processing in Human Brain by Interpreting Machine Learning Models
• Towards Interpretable Deep Extreme Multi-label Learning
• Model-Agnostic Explanations using Minimal Forcing Subsets
• Regression Concept Vectors for Bidirectional Explanations in Histopathology
• Discriminatory Expressions to Produce Interpretable Models in Short Documents
• Understanding Interpretability by generalized distillation in Supervised Classification
• Outlining Traceability: A Principle for Operationalizing Accountability in Computing Systems
• From Receptive to Productive: Learning to Use Confusing Words through Automatically Selected Example Sentences
• Towards Model-informed Precision Dosing with Expert-in-the-loop Machine Learning
• DRILL-- Deep Reinforcement Learning for Refinement Operators in $\mathcal{A}\mathcal{LC}$
• Extending SOUP to ML Models When DesigningCertified Medical Systems
• Connecting Attributions and QA Model Behavior on Realistic Counterfactuals
• Fairness for Whom? Critically reframing fairness with Nash Welfare Product
• Maintaining The Humanity of Our Models
• Axiomatic Interpretability for Multiclass Additive Models
• Semantic bottleneck for computer vision tasks
• Query Understanding via Intent Description Generation
• Model Reduction of Shallow CNN Model for Reliable Deployment of Information Extraction from Medical Reports