XNLI: Explaining and Diagnosing NLI-based Visual Data Analysis
arXiv:2301.10385 · doi:10.1109/TVCG.2023.3240003
Abstract
Natural language interfaces (NLIs) enable users to flexibly specify analytical intentions in data visualization. However, diagnosing the visualization results without understanding the underlying generation process is challenging. Our research explores how to provide explanations for NLIs to help users locate the problems and further revise the queries. We present XNLI, an explainable NLI system for visual data analysis. The system introduces a Provenance Generator to reveal the detailed process of visual transformations, a suite of interactive widgets to support error adjustments, and a Hint Generator to provide query revision hints based on the analysis of user queries and interactions. Two usage scenarios of XNLI and a user study verify the effectiveness and usability of the system. Results suggest that XNLI can significantly enhance task accuracy without interrupting the NLI-based analysis process.
arXiv comment14 pages, 7 figures. A preprint version of a publication at IEEE Transactions on Visualization and Computer Graphics (TVCG), 2023
References in corpus (17)
- NL4DV: A Toolkit for Generating Analytic Specifications for Data Visualization from Natural Language Queries
- Towards Natural Language Interfaces for Data Visualization: A Survey
- FlowSense: A Natural Language Interface for Visual Data Exploration within a Dataflow System
- Towards Natural Language-Based Visualization Authoring
- Touch? Speech? or Touch and Speech? Investigating Multimodal Interaction for Visual Network Exploration and Analysis
- Collecting and Characterizing Natural Language Utterances for Specifying Data Visualizations
- Interweaving Multimodal Interaction with Flexible Unit Visualizations for Data Exploration
- CrossData: Leveraging Text-Data Connections for Authoring Data Documents
- QueryVis: Logic-based diagrams help users understand complicated SQL queries faster
- Persua: A Visual Interactive System to Enhance the Persuasiveness of Arguments in Online Discussion
- Sporthesia: Augmenting Sports Videos Using Natural Language
- Visualizing the Scripts of Data Wrangling with SOMNUS
- RASIPAM: Interactive Pattern Mining of Multivariate Event Sequences in Racket Sports
- DeHumor: Visual Analytics for Decomposing Humor
- Quda: Natural Language Queries for Visual Data Analytics
- nvBench: A Large-Scale Synthesized Dataset for Cross-Domain Natural Language to Visualization Task
- Beyond Interestingness: Semantic and Context-Aware Natural Language Query Recommendations for Visual Data Analysis
Cited by in corpus (7)
- PromptMagician: Interactive Prompt Engineering for Text-to-Image Creation
- WaitGPT: Monitoring and Steering Conversational LLM Agent in Data Analysis with On-the-Fly Code Visualization
- CommonsenseVIS: Visualizing and Understanding Commonsense Reasoning Capabilities of Natural Language Models
- Quantivine: A Visualization Approach for Large-scale Quantum Circuit Representation and Analysis
- Narrative Player: Reviving Data Narratives with Visuals
- JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Models
- SPROUT: an Interactive Authoring Tool for Generating Programming Tutorials with the Visualization of Large Language Models