Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environments
arXiv:2102.08492
Abstract
We study black-box reward poisoning attacks against reinforcement learning (RL), in which an adversary aims to manipulate the rewards to mislead a sequence of RL agents with unknown algorithms to learn a nefarious policy in an environment unknown to the adversary a priori. That is, our attack makes minimum assumptions on the prior knowledge of the adversary: it has no initial knowledge of the environment or the learner, and neither does it observe the learner's internal mechanism except for its performed actions. We design a novel black-box attack, U2, that can provably achieve a near-matching performance to the state-of-the-art white-box attack, demonstrating the feasibility of reward poisoning even in the most challenging black-box setting.
References in corpus (20)
- Trust Region Policy Optimization
- Deep Reinforcement Learning for Page-wise Recommendations
- Deep Reinforcement Learning for Dialogue Generation
- Cooperative Inverse Reinforcement Learning
- Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learning
- Delving into adversarial attacks on deep policies
- Tactics of Adversarial Attack on Deep Reinforcement Learning Agents
- Near-Optimally Teaching the Crowd to Classify
- On Lower Bounds for Regret in Reinforcement Learning
- Adversarial Attacks on Stochastic Bandits
- Policy Poisoning in Batch Reinforcement Learning and Control
- Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learning
- Teaching Inverse Reinforcement Learners via Features and Demonstrations
- Understanding the Role of Adaptivity in Machine Teaching: The Case of Version Space Learners
- Data Poisoning Attacks on Stochastic Bandits
- Reward-Free Exploration for Reinforcement Learning
- Preference-Based Batch and Sequential Teaching: Towards a Unified View of Models
- Deceptive Reinforcement Learning Under Adversarial Manipulations on Cost Signals
- Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamics
- Machine Teaching of Active Sequential Learners