The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations

Abstract

Machine learning (ML) models are nowadays used in complex applications in various domains, such as medicine, bioinformatics, and other sciences. Due to their black box nature, however, it may sometimes be hard to understand and trust the results they provide. This has increased the demand for reliable visualization tools related to enhancing trust in ML models, which has become a prominent topic of research in the visualization community over the past decades. To provide an overview and present the frontiers of current research on the topic, we present a State-of-the-Art Report (STAR) on enhancing trust in ML models with the use of interactive visualization. We define and describe the background of the topic, introduce a categorization for visualization techniques that aim to accomplish this goal, and discuss insights and opportunities for future research directions. Among our contributions is a categorization of trust against different facets of interactive ML, expanded and improved from previous research. Our results are investigated from different analytical perspectives: (a) providing a statistical overview, (b) summarizing key findings, (c) performing topic analyses, and (d) exploring the data sets used in the individual papers, all with the support of an interactive web-based survey browser. We intend this survey to be beneficial for visualization researchers whose interests involve making ML models more trustworthy, as well as researchers and practitioners from other disciplines in their search for effective visualization techniques suitable for solving their tasks with confidence and conveying meaning to their data.

References in corpus (8)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Understanding Neural Networks Through Deep Visualization
• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• Assessment of algorithms for mitosis detection in breast cancer histopathology images
• Do Convolutional Neural Networks Learn Class Hierarchy?
• Interpretable and Steerable Sequence Learning via Prototypes
• Using Visual Analytics to Interpret Predictive Machine Learning Models
• CNNComparator: Comparative Analytics of Convolutional Neural Networks

Cited by in corpus (19)

• From Information to Choice: A Critical Inquiry Into Visualization Tools for Decision Making
• When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System
• TimberTrek: Exploring and Curating Sparse Decision Trees with Interactive Visualization
• Visualization for Trust in Machine Learning Revisited: The State of the Field in 2023
• Visualization Of Class Activation Maps To Explain AI Classification Of Network Packet Captures
• AutoDOViz: Human-Centered Automation for Decision Optimization
• Decoupling Judgment and Decision Making: A Tale of Two Tails
• Visual Analytics for Explainable and Trustworthy Artificial Intelligence
• Towards Efficient Visual Simplification of Computational Graphs in Deep Neural Networks
• DeforestVis: Behavior Analysis of Machine Learning Models with Surrogate Decision Stumps
• Beware of Validation by Eye: Visual Validation of Linear Trends in Scatterplots
• Comparing Native and Non-native English Speakers' Behaviors in Collaborative Writing through Visual Analytics
• Adversarial Attacks on Machine Learning-Aided Visualizations
• Towards the Visualization of Aggregated Class Activation Maps to Analyse the Global Contribution of Class Features
• Measure Utility, Gain Trust: Practical Advice for XAI Researcher
• Towards Visual Explainable Active Learning for Zero-Shot Classification
• Visual Validation versus Visual Estimation: A Study on the Average Value in Scatterplots
• Who should I trust? A Visual Analytics Approach for Comparing Net Load Forecasting Models
• Does the Layout Really Matter? A Study on Visual Model Accuracy Estimation