Progressive Analytics: A Computation Paradigm for Exploratory Data Analysis

Abstract

Exploring data requires a fast feedback loop from the analyst to the system, with a latency below about 10 seconds because of human cognitive limitations. When data becomes large or analysis becomes complex, sequential computations can no longer be completed in a few seconds and data exploration is severely hampered. This article describes a novel computation paradigm called Progressive Computation for Data Analysis or more concisely Progressive Analytics, that brings at the programming language level a low-latency guarantee by performing computations in a progressive fashion. Moving this progressive computation at the language level relieves the programmer of exploratory data analysis systems from implementing the whole analytics pipeline in a progressive way from scratch, streamlining the implementation of scalable exploratory data analysis systems. This article describes the new paradigm through a prototype implementation called ProgressiVis, and explains the requirements it implies through examples.

10 pages

References in corpus (1)

• The NumPy array: a structure for efficient numerical computation

Cited by in corpus (6)

• DV-ARPA: Data Variety Aware Resource Provisioning for Big Data Processing in Accumulative Applications
• InfiniViz: Interactive Visual Exploration using Progressive Bin Refinement
• Visualizing and Analyzing Entity Activity on the Bitcoin Network
• MESSI: In-Memory Data Series Indexing
• Fast Data Series Indexing for In-Memory Data
• Interactive Region-of-Interest Discovery using Exploratory Feedback