Ubiquitous Distributed Deep Reinforcement Learning at the Edge: Analyzing Byzantine Agents in Discrete Action Spaces
arXiv:2008.07863
Abstract
The integration of edge computing in next-generation mobile networks is bringing low-latency and high-bandwidth ubiquitous connectivity to a myriad of cyber-physical systems. This will further boost the increasing intelligence that is being embedded at the edge in various types of autonomous systems, where collaborative machine learning has the potential to play a significant role. This paper discusses some of the challenges in multi-agent distributed deep reinforcement learning that can occur in the presence of byzantine or malfunctioning agents. As the simulation-to-reality gap gets bridged, the probability of malfunctions or errors must be taken into account. We show how wrong discrete actions can significantly affect the collaborative learning effort. In particular, we analyze the effect of having a fraction of agents that might perform the wrong action with a given probability. We study the ability of the system to converge towards a common working policy through the collaborative learning process based on the number of experiences from each of the agents to be aggregated for each policy update, together with the fraction of wrong actions from agents experiencing malfunctions. Our experiments are carried out in a simulation environment using the Atari testbed for the discrete action spaces, and advantage actor-critic (A2C) for the distributed multi-agent training.
Accepted to the 11th International Conference on Emerging Ubiquitous Systems and Pervasive Networks (EUSPN 2020) , Elsevier (2020)
References in corpus (7)
- Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
- Sim-to-Real Reinforcement Learning for Deformable Object Manipulation
- Adversarial Policies: Attacking Deep Reinforcement Learning
- Modeling Others using Oneself in Multi-Agent Reinforcement Learning
- Blockchain-Powered Collaboration in Heterogeneous Swarms of Robots
- DeepRacer: Educational Autonomous Racing Platform for Experimentation with Sim2Real Reinforcement Learning
- Time-Varying Formation Controllers for Unmanned Aerial Vehicles Using Deep Reinforcement Learning