Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight

Abstract

Deep reinforcement learning has shown promising results in learning control policies for complex sequential decision-making tasks. However, these neural network-based policies are known to be vulnerable to adversarial examples. This vulnerability poses a potentially serious threat to safety-critical systems such as autonomous vehicles. In this paper, we propose a defense mechanism to defend reinforcement learning agents from adversarial attacks by leveraging an action-conditioned frame prediction module. Our core idea is that the adversarial examples targeting at a neural network-based policy are not effective for the frame prediction model. By comparing the action distribution produced by a policy from processing the current observed frame to the action distribution produced by the same policy from processing the predicted frame from the action-conditioned frame prediction module, we can detect the presence of adversarial examples. Beyond detecting the presence of adversarial examples, our method allows the agent to continue performing the task using the predicted frame when the agent is under attack. We evaluate the performance of our algorithm using five games in Atari 2600. Our results demonstrate that the proposed defense mechanism achieves favorable performance against baseline algorithms in detecting adversarial examples and in earning rewards when the agents are under attack.

Project page: http://yclin.me/RL_attack_detection/ Code: https://github.com/yenchenlin/rl-attack-detection

References in corpus (18)

• End to End Learning for Self-Driving Cars
• Action-Conditional Video Prediction using Deep Networks in Atari Games
• Decomposing Motion and Content for Natural Video Sequence Prediction
• Adversarial Machine Learning at Scale
• Detecting Adversarial Samples from Artifacts
• On the (Statistical) Detection of Adversarial Examples
• Unsupervised Learning for Physical Interaction through Video Prediction
• MagNet: a Two-Pronged Defense against Adversarial Examples
• Houdini: Fooling Deep Structured Prediction Models
• Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning
• Delving into adversarial attacks on deep policies
• Adversarial Examples for Semantic Segmentation and Object Detection
• Tactics of Adversarial Attack on Deep Reinforcement Learning Agents
• Extending Defensive Distillation
• Feature Squeezing Mitigates and Detects Carlini/Wagner Adversarial Examples
• Adversarial examples for generative models
• Deep Visual Foresight for Planning Robot Motion
• Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics

Cited by in corpus (5)

• Adversarial Examples: Attacks and Defenses for Deep Learning
• Whatever Does Not Kill Deep Reinforcement Learning, Makes It Stronger
• The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement Learning
• Adversarial Reinforcement Learning under Partial Observability in Autonomous Computer Network Defence
• Learning to Cope with Adversarial Attacks