Adversarial Attacks on Neural Network Policies

Abstract

Machine learning classifiers are known to be vulnerable to inputs maliciously constructed by adversaries to force misclassification. Such adversarial examples have been extensively studied in the context of computer vision applications. In this work, we show adversarial attacks are also effective when targeting neural network policies in reinforcement learning. Specifically, we show existing adversarial example crafting techniques can be used to significantly degrade test-time performance of trained policies. Our threat model considers adversaries capable of introducing small perturbations to the raw input of the policy. We characterize the degree of vulnerability across tasks and training algorithms, for a subclass of adversarial-example attacks in white-box and black-box settings. Regardless of the learned task or training algorithm, we observe a significant drop in performance, even with small adversarial perturbations that do not interfere with human perception. Videos are available at http://rll.berkeley.edu/adversarial.

Cited by in corpus (23)

• Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
• Adversarial Policies: Attacking Deep Reinforcement Learning
• Adversarial Examples in Modern Machine Learning: A Review
• Towards Resilient Artificial Intelligence: Survey and Research Issues
• Adversary A3C for Robust Reinforcement Learning
• Action-Manipulation Attacks Against Stochastic Bandits: Attacks and Defense
• Say What I Want: Towards the Dark Side of Neural Dialogue Models
• Optimal Attacks on Reinforcement Learning Policies
• Adversarial Attacks for Tabular Data: Application to Fraud Detection and Imbalanced Data
• Falsification-Based Robust Adversarial Reinforcement Learning
• Online Robustness Training for Deep Reinforcement Learning
• Robustness Testing for Multi-Agent Reinforcement Learning: State Perturbations on Critical Agents
• Robust Predictable Control
• Adversarial Examples on Object Recognition: A Comprehensive Survey
• Towards Security Threats of Deep Learning Systems: A Survey
• AdvGAN++ : Harnessing latent layers for adversary generation
• Online Robust Reinforcement Learning with Model Uncertainty
• Synthesis-guided Adversarial Scenario Generation for Gray-box Feedback Control Systems with Sensing Imperfections
• Policy Smoothing for Provably Robust Reinforcement Learning
• Adversary Agnostic Robust Deep Reinforcement Learning
• An Empirical Review of Adversarial Defenses
• Improving Gradient-based Adversarial Training for Text Classification by Contrastive Learning and Auto-Encoder
• Deep Learning-Based Autonomous Driving Systems: A Survey of Attacks and Defenses