Multi-Agent Deep Reinforcement Learning for Request Dispatching in Distributed-Controller Software-Defined Networking

Abstract

Recently, distributed controller architectures have been quickly gaining popularity in Software-Defined Networking (SDN). However, the use of distributed controllers introduces a new and important Request Dispatching (RD) problem with the goal for every SDN switch to properly dispatch their requests among all controllers so as to optimize network performance. This goal can be fulfilled by designing an RD policy to guide distribution of requests at each switch. In this paper, we propose a Multi-Agent Deep Reinforcement Learning (MA-DRL) approach to automatically design RD policies with high adaptability and performance. This is achieved through a new problem formulation in the form of a Multi-Agent Markov Decision Process (MA-MDP), a new adaptive RD policy design and a new MA-DRL algorithm called MA-PPO. Extensive simulation studies show that our MA-DRL technique can effectively train RD policies to significantly outperform man-made policies, model-based policies, as well as RD policies learned via single-agent DRL algorithms.

References in corpus (11)

• Trust Region Policy Optimization
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• A Survey and Critique of Multiagent Deep Reinforcement Learning
• The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games
• Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents
• Large-Scale Traffic Signal Control Using a Novel Multi-Agent Reinforcement Learning
• CFR-RL: Traffic Engineering with Reinforcement Learning in SDN
• An Adaptive Clipping Approach for Proximal Policy Optimization
• Effective Scheduling Function Design in SDN through Deep Reinforcement Learning
• Policy Regularization via Noisy Advantage Values for Cooperative Multi-agent Actor-Critic methods

Cited by in corpus (1)

• Keep It Simple: Fault Tolerance Evaluation of Federated Learning with Unreliable Clients