Adversarial Policies: Attacking Deep Reinforcement Learning

Abstract

Deep reinforcement learning (RL) policies are known to be vulnerable to adversarial perturbations to their observations, similar to adversarial examples for classifiers. However, an attacker is not usually able to directly modify another agent's observations. This might lead one to wonder: is it possible to attack an RL agent simply by choosing an adversarial policy acting in a multi-agent environment so as to create natural observations that are adversarial? We demonstrate the existence of adversarial policies in zero-sum games between simulated humanoid robots with proprioceptive observations, against state-of-the-art victims trained via self-play to be robust to opponents. The adversarial policies reliably win against the victims but generate seemingly random and uncoordinated behavior. We find that these policies are more successful in high-dimensional environments, and induce substantially different activations in the victim policy network than when the victim plays against a normal opponent. Videos are available at https://adversarialpolicies.github.io/.

Presented at ICLR 2020

References in corpus (10)

• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• Adversarial Examples Are Not Bugs, They Are Features
• Robust Adversarial Reinforcement Learning
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Verifiable Reinforcement Learning via Policy Extraction
• Motivating the Rules of the Game for Adversarial Example Research
• Delving into adversarial attacks on deep policies
• Adversarial Attacks on Neural Network Policies
• Open-ended Learning in Symmetric Zero-sum Games
• On the Geometry of Adversarial Examples

Cited by in corpus (32)

• Robust Reinforcement Learning on State Observations with Learned Optimal Adversary
• Node Injection Attacks on Graphs via Reinforcement Learning
• Towards Resilient Artificial Intelligence: Survey and Research Issues
• Network Environment Design for Autonomous Cyberdefense
• Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RL
• Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamics
• On the Robustness of Cooperative Multi-Agent Reinforcement Learning
• Robust Reinforcement Learning using Adversarial Populations
• One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RL
• Security and Machine Learning in the Real World
• Evaluating the Robustness of Collaborative Agents
• Solving Dual Sourcing Problems with Supply Mode Dependent Failure Rates
• Advancing the Research and Development of Assured Artificial Intelligence and Machine Learning Capabilities
• Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentation
• Network Defense is Not a Game
• Resolving Spurious Correlations in Causal Models of Environments via Interventions
• Synthesis-guided Adversarial Scenario Generation for Gray-box Feedback Control Systems with Sensing Imperfections
• CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing
• Preventing Imitation Learning with Adversarial Policy Ensembles
• Robustness to Adversarial Attacks in Learning-Enabled Controllers
• A Survey on Trust Metrics for Autonomous Robotic Systems
• FormulaZero: Distributionally Robust Online Adaptation via Offline Population Synthesis
• Multi-Task Federated Reinforcement Learning with Adversaries
• Towards Closing the Sim-to-Real Gap in Collaborative Multi-Robot Deep Reinforcement Learning
• Generating Adversarial Disturbances for Controller Verification
• Generating Socially Acceptable Perturbations for Efficient Evaluation of Autonomous Vehicles
• Smoothed Inference for Adversarially-Trained Models
• One Neuron to Fool Them All
• Ubiquitous Distributed Deep Reinforcement Learning at the Edge: Analyzing Byzantine Agents in Discrete Action Spaces
• Adversarial Reinforcement Learning in Dynamic Channel Access and Power Control
• A Game Theoretic Analysis of LQG Control under Adversarial Attack
• A study of first-passage time minimization via Q-learning in heated gridworlds