Internet Congestion Control via Deep Reinforcement Learning

Abstract

We present and investigate a novel and timely application domain for deep reinforcement learning (RL): Internet congestion control. Congestion control is the core networking task of modulating traffic sources' data-transmission rates to efficiently utilize network capacity, and is the subject of extensive attention in light of the advent of Internet services such as live video, virtual reality, Internet-of-Things, and more. We show that casting congestion control as RL enables training deep network policies that capture intricate patterns in data traffic and network conditions, and leverage this to outperform the state-of-the-art. We also highlight significant challenges facing real-world adoption of RL-based congestion control, including fairness, safety, and generalization, which are not trivial to address within conventional RL formalism. To facilitate further research and reproducibility of our results, we present a test suite for RL-guided congestion control based on the OpenAI Gym interface.

arXiv comment10 pages, accepted to ICML 2019

References in corpus (9)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Trust Region Policy Optimization
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Efficient GAN-Based Anomaly Detection
• PCC: Re-architecting Congestion Control for Consistent High Performance
• DARLA: Improving Zero-Shot Transfer in Reinforcement Learning
• Distance-based Confidence Score for Neural Network Classifiers
• Iroko: A Framework to Prototype Reinforcement Learning for Data Center Traffic Control
• Grounding Language for Transfer in Deep Reinforcement Learning

Cited by in corpus (8)

• A Computational Approach to Packet Classification
• Learning Fair Policies in Decentralized Cooperative Multi-Agent Reinforcement Learning
• Neural Packet Classification
• When Machine Learning Meets Congestion Control: A Survey and Comparison
• Escaping from Zero Gradient: Revisiting Action-Constrained Reinforcement Learning via Frank-Wolfe Policy Optimization
• Learning on Abstract Domains: A New Approach for Verifiable Guarantee in Reinforcement Learning
• Towards A Domain-Customized Automated Machine Learning Framework For Networks and Systems
• Guarded Deep Learning using Scenario-Based Modeling