Benchmarking Distributed Stream Processing Platforms for IoT Applications

Abstract

Internet of Things (IoT) is a technology paradigm where millions of sensors monitor, and help inform or manage, physical, envi- ronmental and human systems in real-time. The inherent closed-loop re- sponsiveness and decision making of IoT applications makes them ideal candidates for using low latency and scalable stream processing plat- forms. Distributed Stream Processing Systems (DSPS) are becoming es- sential components of any IoT stack, but the efficacy and performance of contemporary DSPS have not been rigorously studied for IoT data streams and applications. Here, we develop a benchmark suite and per- formance metrics to evaluate DSPS for streaming IoT applications. The benchmark includes 13 common IoT tasks classified across various func- tional categories and forming micro-benchmarks, and two IoT applica- tions for statistical summarization and predictive analytics that leverage various dataflow compositional features of DSPS. These are coupled with stream workloads sourced from real IoT observations from smart cities. We validate the IoT benchmark for the popular Apache Storm DSPS, and present empirical results.

To Appear in VLDB-TPCTC (2016) , 20 pages,Benchmark,IoT,smart cities ,velocity

References in corpus (1)

• Using coarse GPS data to quantify city-scale transportation system resilience to extreme events

Cited by in corpus (7)

• RIoTBench: A Real-time IoT Benchmark for Distributed Stream Processing Platforms
• Efficient Data Perturbation for Privacy Preserving and Accurate Data Stream Mining
• An Efficient and Scalable Privacy Preserving Algorithm for Big Data and Data Streams
• Distributed Scheduling of Event Analytics across Edge and Cloud
• Model-driven Scheduling for Distributed Stream Processing Systems
• ESTemd: A Distributed Processing Framework for Environmental Monitoring based on Apache Kafka Streaming Engine
• A Survey of Benchmarks to Evaluate Data Analytics for Smart-* Applications