The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement Learning

Abstract

Since the inception of Deep Reinforcement Learning (DRL) algorithms, there has been a growing interest in both research and industrial communities in the promising potentials of this paradigm. The list of current and envisioned applications of deep RL ranges from autonomous navigation and robotics to control applications in the critical infrastructure, air traffic control, defense technologies, and cybersecurity. While the landscape of opportunities and the advantages of deep RL algorithms are justifiably vast, the security risks and issues in such algorithms remain largely unexplored. To facilitate and motivate further research on these critical challenges, this paper presents a foundational treatment of the security problem in DRL. We formulate the security requirements of DRL, and provide a high-level threat model through the classification and identification of vulnerabilities, attack vectors, and adversarial capabilities. Furthermore, we present a review of current literature on security of deep RL from both offensive and defensive perspectives. Lastly, we enumerate critical research venues and open problems in mitigation and prevention of intentional attacks against deep RL as a roadmap for further research in this area.

arXiv commentarXiv admin note: text overlap with arXiv:1807.06064, arXiv:1712.03632, arXiv:1803.02811, arXiv:1710.00814 by other authors

References in corpus (25)

• Continuous control with deep reinforcement learning
• Explaining and Harnessing Adversarial Examples
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• Benchmarking Deep Reinforcement Learning for Continuous Control
• Poisoning Attacks against Support Vector Machines
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• Noisy Networks for Exploration
• The Space of Transferable Adversarial Examples
• Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
• Robust Adversarial Reinforcement Learning
• Learning to reinforcement learn
• Detecting Adversarial Samples from Artifacts
• MagNet: a Two-Pronged Defense against Adversarial Examples
• EPOpt: Learning Robust Neural Network Policies Using Model Ensembles
• Delving into adversarial attacks on deep policies
• AI Safety Gridworlds
• Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach
• Feature Squeezing Mitigates and Detects Carlini/Wagner Adversarial Examples
• Whatever Does Not Kill Deep Reinforcement Learning, Makes It Stronger
• Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight
• Online Robust Policy Learning in the Presence of Unknown Adversaries
• Sequential Attacks on Agents for Long-Term Adversarial Goals
• Mitigation of Policy Manipulation Attacks on Deep Q-Networks with Parameter-Space Noise
• Reinforcement Learning for Autonomous Defence in Software-Defined Networking

Cited by in corpus (11)

• Network Environment Design for Autonomous Cyberdefense
• Deceptive Reinforcement Learning Under Adversarial Manipulations on Cost Signals
• Sequential Triggers for Watermarking of Deep Reinforcement Learning Policies
• Analysis and Improvement of Adversarial Training in DQN Agents With Adversarially-Guided Exploration (AGE)
• Network Defense is Not a Game
• Attacking Deep Reinforcement Learning-Based Traffic Signal Control Systems with Colluding Vehicles
• Manipulating Reinforcement Learning: Poisoning Attacks on Cost Signals
• Founding The Domain of AI Forensics
• Multi-Task Federated Reinforcement Learning with Adversaries
• Mitigation of Adversarial Policy Imitation via Constrained Randomization of Policy (CRoP)
• Adversarial Attacks on Deep Algorithmic Trading Policies