The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations

Abstract

Machine learning (ML) models are nowadays used in complex applications in various domains, such as medicine, bioinformatics, and other sciences. Due to their black box nature, however, it may sometimes be hard to understand and trust the results they provide. This has increased the demand for reliable visualization tools related to enhancing trust in ML models, which has become a prominent topic of research in the visualization community over the past decades. To provide an overview and present the frontiers of current research on the topic, we present a State-of-the-Art Report (STAR) on enhancing trust in ML models with the use of interactive visualization. We define and describe the background of the topic, introduce a categorization for visualization techniques that aim to accomplish this goal, and discuss insights and opportunities for future research directions. Among our contributions is a categorization of trust against different facets of interactive ML, expanded and improved from previous research. Our results are investigated from different analytical perspectives: (a) providing a statistical overview, (b) summarizing key findings, (c) performing topic analyses, and (d) exploring the data sets used in the individual papers, all with the support of an interactive web-based survey browser. We intend this survey to be beneficial for visualization researchers whose interests involve making ML models more trustworthy, as well as researchers and practitioners from other disciplines in their search for effective visualization techniques suitable for solving their tasks with confidence and conveying meaning to their data.

References in corpus (13)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Understanding Neural Networks Through Deep Visualization
• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• Assessment of algorithms for mitosis detection in breast cancer histopathology images
• explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning
• Do Convolutional Neural Networks Learn Class Hierarchy?
• FairSight: Visual Analytics for Fairness in Decision Making
• Interpretable and Steerable Sequence Learning via Prototypes
• Using Visual Analytics to Interpret Predictive Machine Learning Models
• RNNbow: Visualizing Learning via Backpropagation Gradients in Recurrent Neural Networks
• FeatureExplorer: Interactive Feature Selection and Exploration of Regression Models for Hyperspectral Images
• CNNComparator: Comparative Analytics of Convolutional Neural Networks
• Visualizing RNN States with Predictive Semantic Encodings

Cited by in corpus (34)

• Post-hoc Interpretability for Neural NLP: A Survey
• StackGenVis: Alignment of Data, Algorithms, and Models for Stacking Ensemble Learning Using Performance Metrics
• A Survey on ML4VIS: Applying Machine Learning Advances to Data Visualization
• VIS30K: A Collection of Figures and Tables from IEEE Visualization Conference Publications
• From Information to Choice: A Critical Inquiry Into Visualization Tools for Decision Making
• FeatureEnVi: Visual Analytics for Feature Engineering Using Stepwise Selection and Semi-Automatic Extraction Approaches
• When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System
• VisEvol: Visual Analytics to Support Hyperparameter Search through Evolutionary Optimization
• TimberTrek: Exploring and Curating Sparse Decision Trees with Interactive Visualization
• Visualizing Graph Neural Networks with CorGIE: Corresponding a Graph to Its Embedding
• A Comparative Analysis of Industry Human-AI Interaction Guidelines
• VisRuler: Visual Analytics for Extracting Decision Rules from Bagged and Boosted Decision Trees
• Visualization for Trust in Machine Learning Revisited: The State of the Field in 2023
• Visualization Of Class Activation Maps To Explain AI Classification Of Network Packet Captures
• Are Metrics Enough? Guidelines for Communicating and Visualizing Predictive Models to Subject Matter Experts
• AutoDOViz: Human-Centered Automation for Decision Optimization
• HardVis: Visual Analytics to Handle Instance Hardness Using Undersampling and Oversampling Techniques
• Decoupling Judgment and Decision Making: A Tale of Two Tails
• Visual Analytics for Explainable and Trustworthy Artificial Intelligence
• Towards Efficient Visual Simplification of Computational Graphs in Deep Neural Networks
• Labeling of Cultural Heritage Collections on the Intersection of Visual Analytics and Digital Humanities
• DeforestVis: Behavior Analysis of Machine Learning Models with Surrogate Decision Stumps
• Document Domain Randomization for Deep Learning Document Layout Extraction
• An Interdisciplinary Perspective on Evaluation and Experimental Design for Visual Text Analytics: Position Paper
• PSEUDo: Interactive Pattern Search in Multivariate Time Series with Locality-Sensitive Hashing and Relevance Feedback
• Comparing Native and Non-native English Speakers' Behaviors in Collaborative Writing through Visual Analytics
• Beware of Validation by Eye: Visual Validation of Linear Trends in Scatterplots
• Adversarial Attacks on Machine Learning-Aided Visualizations
• Towards the Visualization of Aggregated Class Activation Maps to Analyse the Global Contribution of Class Features
• Towards Visual Explainable Active Learning for Zero-Shot Classification
• Visual Validation versus Visual Estimation: A Study on the Average Value in Scatterplots
• Measure Utility, Gain Trust: Practical Advice for XAI Researcher
• Does the Layout Really Matter? A Study on Visual Model Accuracy Estimation
• Who should I trust? A Visual Analytics Approach for Comparing Net Load Forecasting Models