Federated Deep Reinforcement Learning

Abstract

In deep reinforcement learning, building policies of high-quality is challenging when the feature space of states is small and the training data is limited. Despite the success of previous transfer learning approaches in deep reinforcement learning, directly transferring data or models from an agent to another agent is often not allowed due to the privacy of data and/or models in many privacy-aware applications. In this paper, we propose a novel deep reinforcement learning framework to federatively build models of high-quality for agents with consideration of their privacies, namely Federated deep Reinforcement Learning (FedRL). To protect the privacy of data and models, we exploit Gausian differentials on the information shared with each other when updating their local models. In the experiment, we evaluate our FedRL framework in two diverse domains, Grid-world and Text2Action domains, by comparing to various baselines.

arXiv comment9 pages, 5 figures

References in corpus (8)

• Deep Learning with Differential Privacy
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Federated Optimization:Distributed Optimization Beyond the Datacenter
• QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
• Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability
• Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddings
• Privacy Aware Learning

Cited by in corpus (20)

• Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
• On the Convergence of FedAvg on Non-IID Data
• Federated Forest
• Edge Intelligence: Architectures, Challenges, and Applications
• Federated Learning for Healthcare Informatics
• Differentially Private Graph Classification with GNNs
• Resilient Control of Networked Microgrids using Vertical Federated Reinforcement Learning: Designs and Real-Time Test-Bed Validations
• Communication-Efficient Local Decentralized SGD Methods
• The Tradeoff Between Privacy and Accuracy in Anomaly Detection Using Federated XGBoost
• Federated Learning for Distributed Energy-Efficient Resource Allocation
• Network Support for High-performance Distributed Machine Learning
• FedGL: Federated Graph Learning Framework with Global Self-Supervision
• Enhancing Cyber Resilience of Networked Microgrids using Vertical Federated Reinforcement Learning
• Secure Linear Aggregation Using Decentralized Threshold Additive Homomorphic Encryption For Federated Learning
• Empowering the Edge Intelligence by Air-Ground Integrated Federated Learning
• Accurate and Fast Federated Learning via Combinatorial Multi-Armed Bandits
• Iterative Empirical Game Solving via Single Policy Best Response
• Multi-Task Federated Reinforcement Learning with Adversaries
• Provably Efficient Multi-Task Reinforcement Learning with Model Transfer
• Multi-Party Dual Learning