The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations

Abstract

Machine learning (ML) models are nowadays used in complex applications in various domains, such as medicine, bioinformatics, and other sciences. Due to their black box nature, however, it may sometimes be hard to understand and trust the results they provide. This has increased the demand for reliable visualization tools related to enhancing trust in ML models, which has become a prominent topic of research in the visualization community over the past decades. To provide an overview and present the frontiers of current research on the topic, we present a State-of-the-Art Report (STAR) on enhancing trust in ML models with the use of interactive visualization. We define and describe the background of the topic, introduce a categorization for visualization techniques that aim to accomplish this goal, and discuss insights and opportunities for future research directions. Among our contributions is a categorization of trust against different facets of interactive ML, expanded and improved from previous research. Our results are investigated from different analytical perspectives: (a) providing a statistical overview, (b) summarizing key findings, (c) performing topic analyses, and (d) exploring the data sets used in the individual papers, all with the support of an interactive web-based survey browser. We intend this survey to be beneficial for visualization researchers whose interests involve making ML models more trustworthy, as well as researchers and practitioners from other disciplines in their search for effective visualization techniques suitable for solving their tasks with confidence and conveying meaning to their data.

References in corpus (34)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Understanding Neural Networks Through Deep Visualization
• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• Model-Agnostic Interpretability of Machine Learning
• The What-If Tool: Interactive Probing of Machine Learning Models
• Assessment of algorithms for mitosis detection in breast cancer histopathology images
• The paradigm-shift of social spambots: Evidence, theories, and tools for the arms race
• RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records
• explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning
• Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models
• GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentation
• Do Convolutional Neural Networks Learn Class Hierarchy?
• The State of the Art in Integrating Machine Learning into Visual Analytics
• FairSight: Visual Analytics for Fairness in Decision Making
• Supporting Analysis of Dimensionality Reduction Results with Contrastive Learning
• ATMSeer: Increasing Transparency and Controllability in Automated Machine Learning
• Interpretable and Steerable Sequence Learning via Prototypes
• Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analytics
• An Incremental Dimensionality Reduction Method for Visualizing Streaming Multidimensional Data
• A Taxonomy and Library for Visualizing Learned Features in Convolutional Neural Networks
• Uncertainty-Aware Principal Component Analysis
• Using Visual Analytics to Interpret Predictive Machine Learning Models
• Defining Locality for Surrogates in Post-hoc Interpretablity
• RNNbow: Visualizing Learning via Backpropagation Gradients in Recurrent Neural Networks
• A User-based Visual Analytics Workflow for Exploratory Model Analysis
• VASSL: A Visual Analytics Toolkit for Social Spambot Labeling
• FeatureExplorer: Interactive Feature Selection and Exploration of Regression Models for Hyperspectral Images
• The Subspace Voyager: Exploring High-Dimensional Data along a Continuum of Salient 3D Subspaces
• CNNComparator: Comparative Analytics of Convolutional Neural Networks
• Selection Bias Tracking and Detailed Subset Comparison for High-Dimensional Data
• NNVA: Neural Network Assisted Visual Analysis of Yeast Cell Polarization Simulation
• Homology-Preserving Dimensionality Reduction via Manifold Landmarking and Tearing
• ICE: An Interactive Configuration Explorer for High Dimensional Categorical Parameter Spaces
• Visualizing RNN States with Predictive Semantic Encodings

Cited by in corpus (34)

• Post-hoc Interpretability for Neural NLP: A Survey
• StackGenVis: Alignment of Data, Algorithms, and Models for Stacking Ensemble Learning Using Performance Metrics
• A Survey on ML4VIS: Applying Machine Learning Advances to Data Visualization
• VIS30K: A Collection of Figures and Tables from IEEE Visualization Conference Publications
• From Information to Choice: A Critical Inquiry Into Visualization Tools for Decision Making
• FeatureEnVi: Visual Analytics for Feature Engineering Using Stepwise Selection and Semi-Automatic Extraction Approaches
• When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System
• VisEvol: Visual Analytics to Support Hyperparameter Search through Evolutionary Optimization
• TimberTrek: Exploring and Curating Sparse Decision Trees with Interactive Visualization
• Visualizing Graph Neural Networks with CorGIE: Corresponding a Graph to Its Embedding
• A Comparative Analysis of Industry Human-AI Interaction Guidelines
• VisRuler: Visual Analytics for Extracting Decision Rules from Bagged and Boosted Decision Trees
• Visualization for Trust in Machine Learning Revisited: The State of the Field in 2023
• Visualization Of Class Activation Maps To Explain AI Classification Of Network Packet Captures
• Are Metrics Enough? Guidelines for Communicating and Visualizing Predictive Models to Subject Matter Experts
• AutoDOViz: Human-Centered Automation for Decision Optimization
• HardVis: Visual Analytics to Handle Instance Hardness Using Undersampling and Oversampling Techniques
• Decoupling Judgment and Decision Making: A Tale of Two Tails
• Visual Analytics for Explainable and Trustworthy Artificial Intelligence
• Towards Efficient Visual Simplification of Computational Graphs in Deep Neural Networks
• Labeling of Cultural Heritage Collections on the Intersection of Visual Analytics and Digital Humanities
• DeforestVis: Behavior Analysis of Machine Learning Models with Surrogate Decision Stumps
• Document Domain Randomization for Deep Learning Document Layout Extraction
• An Interdisciplinary Perspective on Evaluation and Experimental Design for Visual Text Analytics: Position Paper
• PSEUDo: Interactive Pattern Search in Multivariate Time Series with Locality-Sensitive Hashing and Relevance Feedback
• Comparing Native and Non-native English Speakers' Behaviors in Collaborative Writing through Visual Analytics
• Beware of Validation by Eye: Visual Validation of Linear Trends in Scatterplots
• Adversarial Attacks on Machine Learning-Aided Visualizations
• Towards the Visualization of Aggregated Class Activation Maps to Analyse the Global Contribution of Class Features
• Towards Visual Explainable Active Learning for Zero-Shot Classification
• Visual Validation versus Visual Estimation: A Study on the Average Value in Scatterplots
• Measure Utility, Gain Trust: Practical Advice for XAI Researcher
• Does the Layout Really Matter? A Study on Visual Model Accuracy Estimation
• Who should I trust? A Visual Analytics Approach for Comparing Net Load Forecasting Models