Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models

Abstract

Interpretation and diagnosis of machine learning models have gained renewed interest in recent years with breakthroughs in new approaches. We present Manifold, a framework that utilizes visual analysis techniques to support interpretation, debugging, and comparison of machine learning models in a more transparent and interactive manner. Conventional techniques usually focus on visualizing the internal logic of a specific model type (i.e., deep neural networks), lacking the ability to extend to a more complex scenario where different model types are integrated. To this end, Manifold is designed as a generic framework that does not rely on or access the internal logic of the model and solely observes the input (i.e., instances or features) and the output (i.e., the predicted result and probability distribution). We describe the workflow of Manifold as an iterative process consisting of three major phases that are commonly involved in the model development and diagnosis process: inspection (hypothesis), explanation (reasoning), and refinement (verification). The visual components supporting these tasks include a scatterplot-based visual summary that overviews the models' outcome and a customizable tabular view that reveals feature discrimination. We demonstrate current applications of the framework on the classification and regression tasks and discuss other potential machine learning use scenarios where Manifold can be applied.

References in corpus (4)

• Scikit-learn: Machine Learning in Python
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Understanding Neural Networks Through Deep Visualization
• Deck.gl: Large-scale Web-based Visual Analytics Made Easy

Cited by in corpus (47)

• The What-If Tool: Interactive Probing of Machine Learning Models
• explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning
• Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs
• The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations
• StackGenVis: Alignment of Data, Algorithms, and Models for Stacking Ensemble Learning Using Performance Metrics
• FairSight: Visual Analytics for Fairness in Decision Making
• ATMSeer: Increasing Transparency and Controllability in Automated Machine Learning
• Explainable Artificial Intelligence: a Systematic Review
• Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analytics
• Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels
• ConfusionFlow: A model-agnostic visualization for temporal analysis of classifier confusion
• Facilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models
• Diagnosing Concept Drift with Visual Analytics
• VisEvol: Visual Analytics to Support Hyperparameter Search through Evolutionary Optimization
• NNVA: Neural Network Assisted Visual Analysis of Yeast Cell Polarization Simulation
• Class-constrained t-SNE: Combining Data Features and Class Probabilities
• Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML
• Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience
• iSEA: An Interactive Pipeline for Semantic Error Analysis of NLP Models
• VisRuler: Visual Analytics for Extracting Decision Rules from Bagged and Boosted Decision Trees
• ViCE: Visual Counterfactual Explanations for Machine Learning Models
• A Survey of Visual Analytics Techniques for Machine Learning
• The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models
• Under the Hood of Neural Networks: Characterizing Learned Representations by Functional Neuron Populations and Network Ablations
• The Validity, Generalizability and Feasibility of Summative Evaluation Methods in Visual Analytics
• Black-box Error Diagnosis in Deep Neural Networks for Computer Vision: a Survey of Tools
• AHMoSe: A Knowledge-Based Visual Support System for Selecting Regression Machine Learning Models
• Should We Trust (X)AI? Design Dimensions for Structured Experimental Evaluations
• Don't Explain without Verifying Veracity: An Evaluation of Explainable AI with Video Activity Recognition
• SUBPLEX: Towards a Better Understanding of Black Box Model Explanations at the Subpopulation Level
• TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing
• TBSSvis: Visual Analytics for Temporal Blind Source Separation
• NAS-Navigator: Visual Steering for Explainable One-Shot Deep Neural Network Synthesis
• M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis
• HypoML: Visual Analysis for Hypothesis-based Evaluation of Machine Learning Models
• Divisi: Interactive Search and Visualization for Scalable Exploratory Subgroup Analysis
• Model-agnostic interpretation by visualization of feature perturbations
• A Visual Analytics System for Multi-model Comparison on Clinical Data Predictions
• HypperSteer: Hypothetical Steering and Data Perturbation in Sequence Prediction with Deep Learning
• IKIWISI: An Interactive Visual Pattern Generator for Evaluating the Reliability of Vision-Language Models Without Ground Truth
• Unlearning Comparator: A Visual Analytics System for Comparative Evaluation of Machine Unlearning Methods
• An Information-theoretic Visual Analysis Framework for Convolutional Neural Networks
• Towards the Visualization of Aggregated Class Activation Maps to Analyse the Global Contribution of Class Features
• Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks
• Trinary Tools for Continuously Valued Binary Classifiers
• Design and Implementation of the Transparent, Interpretable, and Multimodal (TIM) AR Personal Assistant
• Explainable Adversarial Attacks in Deep Neural Networks Using Activation Profiles