A Survey on ML4VIS: Applying Machine Learning Advances to Data Visualization

Abstract

Inspired by the great success of machine learning (ML), researchers have applied ML techniques to visualizations to achieve a better design, development, and evaluation of visualizations. This branch of studies, known as ML4VIS, is gaining increasing research attention in recent years. To successfully adapt ML techniques for visualizations, a structured understanding of the integration of ML4VISis needed. In this paper, we systematically survey 88 ML4VIS studies, aiming to answer two motivating questions: "what visualization processes can be assisted by ML?" and "how ML techniques can be used to solve visualization problems?" This survey reveals seven main processes where the employment of ML techniques can benefit visualizations:Data Processing4VIS, Data-VIS Mapping, InsightCommunication, Style Imitation, VIS Interaction, VIS Reading, and User Profiling. The seven processes are related to existing visualization theoretical models in an ML4VIS pipeline, aiming to illuminate the role of ML-assisted visualization in general visualizations.Meanwhile, the seven processes are mapped into main learning tasks in ML to align the capabilities of ML with the needs in visualization. Current practices and future opportunities of ML4VIS are discussed in the context of the ML4VIS pipeline and the ML-VIS mapping. While more studies are still needed in the area of ML4VIS, we hope this paper can provide a stepping-stone for future exploration. A web-based interactive browser of this survey is available at https://ml4vis.github.io

arXiv comment19 pages, 12 figures, 4 tables

References in corpus (25)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• NIPS 2016 Tutorial: Generative Adversarial Networks
• explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• The State of the Art in Integrating Machine Learning into Visual Analytics
• The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations
• Learning Visual Importance for Graphic Designs and Data Visualizations
• FlowSense: A Natural Language Interface for Visual Data Exploration within a Dataflow System
• FigureQA: An Annotated Figure Dataset for Visual Reasoning
• InSituNet: Deep Image Synthesis for Parameter Space Exploration of Ensemble Simulations
• Interpretable and Steerable Sequence Learning via Prototypes
• Towards Automated Infographic Design: Deep Learning-based Auto-Extraction of Extensible Timeline
• A Generative Model for Volume Rendering
• What Would a Graph Look Like in This Layout? A Machine Learning Approach to Large Graph Visualization
• An Incremental Dimensionality Reduction Method for Visualizing Streaming Multidimensional Data
• A Deep Generative Model for Graph Layout
• LassoNet: Deep Lasso-Selection of 3D Point Clouds
• Evaluating the Readability of Force Directed Graph Layouts: A Deep Learning Approach
• A Natural-language-based Visual Query Approach of Uncertain Human Trajectories
• Understanding Infographics through Textual and Visual Tag Prediction
• NNVA: Neural Network Assisted Visual Analysis of Yeast Cell Polarization Simulation
• Beyond Heuristics: Learning Visualization Design
• Toward automatic comparison of visualization techniques: Application to graph visualization
• Text-to-Viz: Automatic Generation of Infographics from Proportion-Related Natural Language Statements
• Survey on Visual Analysis of Event Sequence Data

Cited by in corpus (15)

• Towards Natural Language Interfaces for Data Visualization: A Survey
• Structure-aware Visualization Retrieval
• LightVA: Lightweight Visual Analytics with LLM Agent-Based Task Planning and Execution
• DMiner: Dashboard Design Mining and Recommendation
• DeepVATS: Deep Visual Analytics for Time Series
• The State of the Art in Creating Visualization Corpora for Automated Chart Analysis
• AutoDOViz: Human-Centered Automation for Decision Optimization
• Agnostic Visual Recommendation Systems: Open Challenges and Future Directions
• LLM-Assisted Visual Analytics: Opportunities and Challenges
• Reinforced Labels: Multi-Agent Deep Reinforcement Learning for Point-Feature Label Placement
• DeepVIS: Bridging Natural Language and Data Visualization Through Step-wise Reasoning
• Adversarial Attacks on Machine Learning-Aided Visualizations
• Evaluating Graphical Perception Capabilities of Vision Transformers
• Automating the Path: An R&D Agenda for Human-Centered AI and Visualization
• Seeing Eye to AI? Applying Deep-Feature-Based Similarity Metrics to Information Visualization