Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values

Abstract

Machine learning (ML) interpretability techniques can reveal undesirable patterns in data that models exploit to make predictions--potentially causing harms once deployed. However, how to take action to address these patterns is not always clear. In a collaboration between ML and human-computer interaction researchers, physicians, and data scientists, we develop GAM Changer, the first interactive system to help domain experts and data scientists easily and responsibly edit Generalized Additive Models (GAMs) and fix problematic patterns. With novel interaction techniques, our tool puts interpretability into action--empowering users to analyze, validate, and align model behaviors with their knowledge and values. Physicians have started to use our tool to investigate and fix pneumonia and sepsis risk prediction models, and an evaluation with 7 data scientists working in diverse domains highlights that our tool is easy to use, meets their model editing needs, and fits into their current workflows. Built with modern web technologies, our tool runs locally in users' web browsers or computational notebooks, lowering the barrier to use. GAM Changer is available at the following public demo link: https://interpret.ml/gam-changer.

arXiv commentAccepted at KDD 2022. 11 pages, 19 figures. For a demo video, see https://youtu.be/D6whtfInqTc. For a live demo, visit https://interpret.ml/gam-changer

References in corpus (8)

• The What-If Tool: Interactive Probing of Machine Learning Models
• Understanding the Role of Individual Units in a Deep Neural Network
• InterpretML: A Unified Framework for Machine Learning Interpretability
• Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs
• Interpretable Classification Models for Recidivism Prediction
• Finding Experts in Transformer Models
• GAM Changer: Editing Generalized Additive Models with Interactive Visualization
• In Pursuit of Interpretable, Fair and Accurate Machine Learning for Criminal Recidivism Prediction

Cited by in corpus (12)

• Farsight: Fostering Responsible AI Awareness During AI Application Prototyping
• EXMOS: Explanatory Model Steering Through Multifaceted Explanations and Data Configurations
• Misty: UI Prototyping Through Interactive Conceptual Blending
• GAM Coach: Towards Interactive and User-centered Algorithmic Recourse
• SuperNOVA: Design Strategies and Opportunities for Interactive Visualization in Computational Notebooks
• Lessons from the Development of an Anomaly Detection Interface on the Mars Perseverance Rover using the ISHMAP Framework
• MeMemo: On-device Retrieval Augmentation for Private and Personalized Text Generation
• An Explanatory Model Steering System for Collaboration between Domain Experts and AI
• AI, Meet Human: Learning Paradigms for Hybrid Decision Making Systems
• DeforestVis: Behavior Analysis of Machine Learning Models with Surrogate Decision Stumps
• WebSHAP: Towards Explaining Any Machine Learning Models Anywhere
• VIGMA: An Open-Access Framework for Visual Gait and Motion Analytics