Locally Private Distributed Reinforcement Learning

Abstract

We study locally differentially private algorithms for reinforcement learning to obtain a robust policy that performs well across distributed private environments. Our algorithm protects the information of local agents' models from being exploited by adversarial reverse engineering. Since a local policy is strongly being affected by the individual environment, the output of the agent may release the private information unconsciously. In our proposed algorithm, local agents update the model in their environments and report noisy gradients designed to satisfy local differential privacy (LDP) that gives a rigorous local privacy guarantee. By utilizing a set of reported noisy gradients, a central aggregator updates its model and delivers it to different local agents. In our empirical evaluation, we demonstrate how our method performs well under LDP. To the best of our knowledge, this is the first work that actualizes distributed reinforcement learning under LDP. This work enables us to obtain a robust agent that performs well across distributed private environments.

References in corpus (5)

• Massively Parallel Methods for Deep Reinforcement Learning
• Robust Adversarial Reinforcement Learning
• Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving
• EPOpt: Learning Robust Neural Network Policies Using Model Ensembles
• Comparing Population Means under Local Differential Privacy: with Significance and Power

Cited by in corpus (4)

• A Comprehensive Survey on Local Differential Privacy Toward Data Statistics and Analysis
• Local Differential Privacy and Its Applications: A Comprehensive Survey
• ORL-AUDITOR: Dataset Auditing in Offline Deep Reinforcement Learning
• Local Differential Privacy for Regret Minimization in Reinforcement Learning