The What-If Tool: Interactive Probing of Machine Learning Models

Abstract

A key challenge in developing and deploying Machine Learning (ML) systems is understanding their performance across a wide range of inputs. To address this challenge, we created the What-If Tool, an open-source application that allows practitioners to probe, visualize, and analyze ML systems, with minimal coding. The What-If Tool lets practitioners test performance in hypothetical situations, analyze the importance of different data features, and visualize model behavior across multiple models and subsets of input data. It also lets practitioners measure systems according to multiple ML fairness metrics. We describe the design of the tool, and report on real-life usage at different organizations.

IEEE VIS (VAST) 2019

References in corpus (6)

• Towards A Rigorous Science of Interpretable Machine Learning
• Axiomatic Attribution for Deep Networks
• Equality of Opportunity in Supervised Learning
• SmoothGrad: removing noise by adding noise
• AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias
• Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models

Cited by in corpus (92)

• Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges
• Fast Parallel Hypertree Decompositions in Logarithmic Recursion Depth
• The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations
• Explainable AI over the Internet of Things (IoT): Overview, State-of-the-Art and Future Directions
• Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs
• Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits
• Model-Agnostic Counterfactual Explanations for Consequential Decisions
• EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteria
• Supporting Human-AI Collaboration in Auditing LLMs with LLMs
• Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions
• Deliberating with AI: Improving Decision-Making for the Future through Participatory AI Design and Stakeholder Deliberation
• Value Cards: An Educational Toolkit for Teaching Social Impacts of Machine Learning through Deliberation
• ConfusionFlow: A model-agnostic visualization for temporal analysis of classifier confusion
• Farsight: Fostering Responsible AI Awareness During AI Application Prototyping
• Why Fairness Cannot Be Automated: Bridging the Gap Between EU Non-Discrimination Law and AI
• Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning
• Symphony: Composing Interactive Interfaces for Machine Learning
• A survey of algorithmic recourse: definitions, formulations, solutions, and prospects
• Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Survey
• Outcome-Explorer: A Causality Guided Interactive Visual Interface for Interpretable Algorithmic Decision Making
• Directive Explanations for Monitoring the Risk of Diabetes Onset: Introducing Directive Data-Centric Explanations and Combinations to Support What-If Explorations
• Diagnosing Concept Drift with Visual Analytics
• The Grammar of Interactive Explanatory Model Analysis
• Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloud
• A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations
• RECAST: Enabling User Recourse and Interpretability of Toxicity Detection Models with Interactive Visualization
• Explainable Machine Learning with Prior Knowledge: An Overview
• dalex: Responsible Machine Learning with Interactive Explainability and Fairness in Python
• Interactive Debugging and Steering of Multi-Agent AI Systems
• Improving Visualization Interpretation Using Counterfactuals
• StickyLand: Breaking the Linear Presentation of Computational Notebooks
• Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values
• Rational Shapley Values
• Angler: Helping Machine Translation Practitioners Prioritize Model Improvements
• "What We Can't Measure, We Can't Understand": Challenges to Demographic Data Procurement in the Pursuit of Fairness
• Coalitional strategies for efficient individual prediction explanation
• Misty: UI Prototyping Through Interactive Conceptual Blending
• Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications
• Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience
• CommonsenseVIS: Visualizing and Understanding Commonsense Reasoning Capabilities of Natural Language Models
• My Model is Unfair, Do People Even Care? Visual Design Affects Trust and Perceived Bias in Machine Learning
• GAM Coach: Towards Interactive and User-centered Algorithmic Recourse
• A Comparative Analysis of Industry Human-AI Interaction Guidelines
• BeFair: Addressing Fairness in the Banking Sector
• A Survey of Visual Analytics Techniques for Machine Learning
• GeCo: Quality Counterfactual Explanations in Real Time
• The "Who", "What", and "How" of Responsible AI Governance: A Systematic Review and Meta-Analysis of (Actor, Stage)-Specific Tools
• The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models
• How Learner Control and Explainable Learning Analytics on Skill Mastery Shape Student Desires to Finish and Avoid Loss in Tutored Practice
• Towards Directive Explanations: Crafting Explainable AI Systems for Actionable Human-AI Interactions
• MiMICRI: Towards Domain-centered Counterfactual Explanations of Cardiovascular Image Classification Models
• SuperNOVA: Design Strategies and Opportunities for Interactive Visualization in Computational Notebooks
• Melody: Generating and Visualizing Machine Learning Model Summary to Understand Data and Classifiers Together
• DECE: Decision Explorer with Counterfactual Explanations for Machine Learning Models
• Fairness On The Ground: Applying Algorithmic Fairness Approaches to Production Systems
• An Empirical Study of Counterfactual Visualization to Support Visual Causal Inference
• JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Models
• TBSSvis: Visual Analytics for Temporal Blind Source Separation
• Talaria: Interactively Optimizing Machine Learning Models for Efficient Inference
• Beyond Correlation: Incorporating Counterfactual Guidance to Better Support Exploratory Visual Analysis
• Understanding the Dataset Practitioners Behind Large Language Model Development
• Designing Resource Allocation Tools to Promote Fair Allocation: Do Visualization and Information Framing Matter?
• On the Morality of Artificial Intelligence
• M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis
• WebSHAP: Towards Explaining Any Machine Learning Models Anywhere
• FairComp: Workshop on Fairness and Robustness in Machine Learning for Ubiquitous Computing
• AI-Spectra: A Visual Dashboard for Model Multiplicity to Enhance Informed and Transparent Decision-Making
• Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships
• Help or Hinder? Evaluating the Impact of Fairness Metrics and Algorithms in Visualizations for Consensus Ranking
• LangFair: A Python Package for Assessing Bias and Fairness in Large Language Model Use Cases
• Model-agnostic and Scalable Counterfactual Explanations via Reinforcement Learning
• A Visual Analytics Approach for Exploratory Causal Analysis: Exploration, Validation, and Applications
• Clarify: Improving Model Robustness With Natural Language Corrections
• VBridge: Connecting the Dots Between Features and Data to Explain Healthcare Models
• Augmenting Decision Making via Interactive What-If Analysis
• Towards a multi-stakeholder value-based assessment framework for algorithmic systems
• The State of Algorithmic Fairness in Mobile Human-Computer Interaction
• A Visual Analytics Framework for Explaining and Diagnosing Transfer Learning Processes
• Trinary Tools for Continuously Valued Binary Classifiers
• Explanatory Pluralism in Explainable AI
• Measure Utility, Gain Trust: Practical Advice for XAI Researcher
• Nishpaksh: TEC Standard-Compliant Framework for Fairness Auditing and Certification of AI Models
• Cause and Effect: Hierarchical Concept-based Explanation of Neural Networks
• Auditing the Sensitivity of Graph-based Ranking with Visual Analytics
• Explainability Assistant: A Conversational XAI Interface for Interpreting Energy Consumption Models
• A Visual Analytics Approach to Debugging Cooperative, Autonomous Multi-Robot Systems' Worldviews
• Revisiting the Modifiable Areal Unit Problem in Deep Traffic Prediction with Visual Analytics
• Interactive Probing of Multivariate Time Series Prediction Models: A Case of Freight Rate Analysis
• A Sociotechnical View of Algorithmic Fairness
• Automated Dependence Plots
• Opening the Black Box of 3D Reconstruction Error Analysis with VECTOR
• Navigating Time's Possibilities: Plausible Counterfactual Explanations for Multivariate Time-Series Forecast through Genetic Algorithms