Sprintz: Time Series Compression for the Internet of Things

Abstract

Thanks to the rapid proliferation of connected devices, sensor-generated time series constitute a large and growing portion of the world's data. Often, this data is collected from distributed, resource-constrained devices and centralized at one or more servers. A key challenge in this setup is reducing the size of the transmitted data without sacrificing its quality. Lower quality reduces the data's utility, but smaller size enables both reduced network and storage costs at the servers and reduced power consumption in sensing devices. A natural solution is to compress the data at the sensing devices. Unfortunately, existing compression algorithms either violate the memory and latency constraints common for these devices or, as we show experimentally, perform poorly on sensor-generated time series. We introduce a time series compression algorithm that achieves state-of-the-art compression ratios while requiring less than 1KB of memory and adding virtually no latency. This method is suitable not only for low-power devices collecting data, but also for servers storing and querying data; in the latter context, it can decompress at over 3GB/s in a single thread, even faster than many algorithms with much lower compression ratios. A key component of our method is a high-speed forecasting algorithm that can be trained online and significantly outperforms alternatives such as delta coding. Extensive experiments on datasets from many domains show that these results hold not only for sensor data but also across a wide array of other time series.

References in corpus (5)

• Decoding billions of integers per second through vectorization
• Asymmetric numeral systems: entropy coding combining speed of Huffman coding with compression rate of arithmetic coding
• A General SIMD-based Approach to Accelerating Compression Algorithms
• A Better Alternative to Piecewise Linear Time Series Segmentation
• Compressed Sensing of Multi-Channel EEG Signals: The Simultaneous Cosparsity and Low Rank Optimization

Cited by in corpus (6)

• Time series compression: a survey
• What is the State of Neural Network Pruning?
• Scalable Model-Based Management of Correlated Dimensional Time Series in ModelarDB+
• CStream: Parallel Data Stream Compression on Multicore Edge Devices
• Neuromorphic Vision Data Coding: Classifying and Reviewing the Literature
• Graph Spectral Embedding for Parsimonious Transmission of Multivariate Time Series