Whatever Does Not Kill Deep Reinforcement Learning, Makes It Stronger

Abstract

Recent developments have established the vulnerability of deep Reinforcement Learning (RL) to policy manipulation attacks via adversarial perturbations. In this paper, we investigate the robustness and resilience of deep RL to training-time and test-time attacks. Through experimental results, we demonstrate that under noncontiguous training-time attacks, Deep Q-Network (DQN) agents can recover and adapt to the adversarial conditions by reactively adjusting the policy. Our results also show that policies learned under adversarial perturbations are more robust to test-time attacks. Furthermore, we compare the performance of $\epsilon$-greedy and parameter-space noise exploration methods in terms of robustness and resilience against adversarial perturbations.

arXiv admin note: text overlap with arXiv:1701.04143

References in corpus (6)

• Noisy Networks for Exploration
• Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
• Parameter Space Noise for Exploration
• Delving into adversarial attacks on deep policies
• Practical Black-Box Attacks against Machine Learning
• Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight

Cited by in corpus (15)

• A Search-Based Testing Approach for Deep Reinforcement Learning Agents
• Robust Reinforcement Learning on State Observations with Learned Optimal Adversary
• Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations
• Deep reinforcement learning in World-Earth system models to discover sustainable management strategies
• The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement Learning
• Enhancing the Robustness of QMIX against State-adversarial Attacks
• Online Robustness Training for Deep Reinforcement Learning
• Defense Against Reward Poisoning Attacks in Reinforcement Learning
• Analysis and Improvement of Adversarial Training in DQN Agents With Adversarially-Guided Exploration (AGE)
• Coordination-driven learning in multi-agent problem spaces
• CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing
• Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning
• Learning to Cope with Adversarial Attacks
• Adversary Agnostic Robust Deep Reinforcement Learning
• Adversarial Attacks on Deep Algorithmic Trading Policies