Model-Free Control for Distributed Stream Data Processing using Deep Reinforcement Learning

Abstract

In this paper, we focus on general-purpose Distributed Stream Data Processing Systems (DSDPSs), which deal with processing of unbounded streams of continuous data at scale distributedly in real or near-real time. A fundamental problem in a DSDPS is the scheduling problem with the objective of minimizing average end-to-end tuple processing time. A widely-used solution is to distribute workload evenly over machines in the cluster in a round-robin manner, which is obviously not efficient due to lack of consideration for communication delay. Model-based approaches do not work well either due to the high complexity of the system environment. We aim to develop a novel model-free approach that can learn to well control a DSDPS from its experience rather than accurate and mathematically solvable system models, just as a human learns a skill (such as cooking, driving, swimming, etc). Specifically, we, for the first time, propose to leverage emerging Deep Reinforcement Learning (DRL) for enabling model-free control in DSDPSs; and present design, implementation and evaluation of a novel and highly effective DRL-based control framework, which minimizes average end-to-end tuple processing time by jointly learning the system environment via collecting very limited runtime statistics data and making decisions under the guidance of powerful Deep Neural Networks. To validate and evaluate the proposed framework, we implemented it based on a widely-used DSDPS, Apache Storm, and tested it with three representative applications. Extensive experimental results show 1) Compared to Storm's default scheduler and the state-of-the-art model-based method, the proposed framework reduces average tuple processing by 33.5% and 14.0% respectively on average. 2) The proposed framework can quickly reach a good scheduling solution during online learning, which justifies its practicability for online control in DSDPSs.

arXiv comment14 pages, this paper has been accepted by VLDB 2018

References in corpus (8)

• Continuous control with deep reinforcement learning
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Benchmarking Deep Reinforcement Learning for Continuous Control
• Deep Reinforcement Learning in Large Discrete Action Spaces
• Language Understanding for Text-based Games Using Deep Reinforcement Learning
• A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning
• Muppet: MapReduce-Style Processing of Fast Data
• Resource Usage Estimation of Data Stream Processing Workloads in Datacenter Clouds

Cited by in corpus (3)

• Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
• Fine-Grained Modeling and Optimization for Intelligent Resource Management in Big Data Processing
• Zero-Shot Cost Models for Distributed Stream Processing