FlowSense: A Natural Language Interface for Visual Data Exploration within a Dataflow System
arXiv:1908.00681 · doi:10.1109/TVCG.2019.2934668
Abstract
Dataflow visualization systems enable flexible visual data exploration by allowing the user to construct a dataflow diagram that composes query and visualization modules to specify system functionality. However learning dataflow diagram usage presents overhead that often discourages the user. In this work we design FlowSense, a natural language interface for dataflow visualization systems that utilizes state-of-the-art natural language processing techniques to assist dataflow diagram construction. FlowSense employs a semantic parser with special utterance tagging and special utterance placeholders to generalize to different datasets and dataflow diagrams. It explicitly presents recognized dataset and diagram special utterances to the user for dataflow context awareness. With FlowSense the user can expand and adjust dataflow diagrams more conveniently via plain English. We apply FlowSense to the VisFlow subset-flow visualization system to enhance its usability. We evaluate FlowSense by one case study with domain experts on a real-world data analysis problem and a formal user study.
Published in IEEE Transactions on Visualization and Computer Graphics
References in corpus (4)
Cited by in corpus (22)
- NL4DV: A Toolkit for Generating Analytic Specifications for Data Visualization from Natural Language Queries
- Towards Natural Language Interfaces for Data Visualization: A Survey
- A Survey on ML4VIS: Applying Machine Learning Advances to Data Visualization
- ChartGPT: Leveraging LLMs to Generate Charts from Abstract Natural Language
- Towards Natural Language-Based Visualization Authoring
- Touch? Speech? or Touch and Speech? Investigating Multimodal Interaction for Visual Network Exploration and Analysis
- Data@Hand: Fostering Visual Exploration of Personal Data on Smartphones Leveraging Speech and Touch Interaction
- Collecting and Characterizing Natural Language Utterances for Specifying Data Visualizations
- XNLI: Explaining and Diagnosing NLI-based Visual Data Analysis
- LightVA: Lightweight Visual Analytics with LLM Agent-Based Task Planning and Execution
- Natural Language Dataset Generation Framework for Visualizations Powered by Large Language Models
- RASIPAM: Interactive Pattern Mining of Multivariate Event Sequences in Racket Sports
- Quda: Natural Language Queries for Visual Data Analytics
- Narrative Player: Reviving Data Narratives with Visuals
- Words of Estimative Correlation: Studying Verbalizations of Scatterplots
- Causal Perception in Question-Answering Systems
- sMolBoxes: Dataflow Model for Molecular Dynamics Exploration
- DATAWEAVER: Authoring Data-Driven Narratives through the Integrated Composition of Visualization and Text
- DeepVIS: Bridging Natural Language and Data Visualization Through Step-wise Reasoning
- Q4EDA: A Novel Strategy for Textual Information Retrieval Based on User Interactions with Visual Representations of Time Series
- Natural Language Interaction for Editing Visual Knowledge Graphs
- AI4VIS: Survey on Artificial Intelligence Approaches for Data Visualization