Analysis and Improvement of Adversarial Training in DQN Agents With Adversarially-Guided Exploration (AGE)

Abstract

This paper investigates the effectiveness of adversarial training in enhancing the robustness of Deep Q-Network (DQN) policies to state-space perturbations. We first present a formal analysis of adversarial training in DQN agents and its performance with respect to the proportion of adversarial perturbations to nominal observations used for training. Next, we consider the sample-inefficiency of current adversarial training techniques, and propose a novel Adversarially-Guided Exploration (AGE) mechanism based on a modified hybrid of the $\epsilon$-greedy algorithm and Boltzmann exploration. We verify the feasibility of this exploration mechanism through experimental evaluation of its performance in comparison with the traditional decaying $\epsilon$-greedy and parameter-space noise exploration algorithms.

References in corpus (5)

• Robust Adversarial Reinforcement Learning
• Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
• A Dissection of Overfitting and Generalization in Continuous Reinforcement Learning
• Whatever Does Not Kill Deep Reinforcement Learning, Makes It Stronger
• The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement Learning