Data Poisoning Attacks on Stochastic Bandits

Abstract

Stochastic multi-armed bandits form a class of online learning problems that have important applications in online recommendation systems, adaptive medical treatment, and many others. Even though potential attacks against these learning algorithms may hijack their behavior, causing catastrophic loss in real-world applications, little is known about adversarial attacks on bandit algorithms. In this paper, we propose a framework of offline attacks on bandit algorithms and study convex optimization based attacks on several popular bandit algorithms. We show that the attacker can force the bandit algorithm to pull a target arm with high probability by a slight manipulation of the rewards in the data. Then we study a form of online attacks on bandit algorithms and propose an adaptive attack strategy against any bandit algorithm without the knowledge of the bandit algorithm. Our adaptive attack strategy can hijack the behavior of the bandit algorithm to suffer a linear regret with only a logarithmic cost to the attacker. Our results demonstrate a significant security threat to stochastic bandits.

Accepted by ICML 2019

Cited by in corpus (22)

• Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learning
• Adaptive Reward-Poisoning Attacks against Reinforcement Learning
• Poisoning Attack against Estimating from Pairwise Comparisons
• Action-Manipulation Attacks Against Stochastic Bandits: Attacks and Defense
• Provably Efficient Black-Box Action Poisoning Attacks Against Reinforcement Learning
• Towards Security Threats of Deep Learning Systems: A Survey
• Policy Teaching in Reinforcement Learning via Environment Poisoning Attacks
• Stochastic Linear Bandits Robust to Adversarial Attacks
• Robust Multi-Agent Multi-Armed Bandits
• Improved Corruption Robust Algorithms for Episodic Reinforcement Learning
• Robust Stochastic Linear Contextual Bandits Under Adversarial Attacks
• Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environments
• Linear Contextual Bandits with Adversarial Corruptions
• Probabilistic Sequential Shrinking: A Best Arm Identification Algorithm for Stochastic Bandits with Corruptions
• Exploiting Heterogeneity in Robust Federated Best-Arm Identification
• Disturbing Reinforcement Learning Agents with Corrupted Rewards
• On Optimal Robustness to Adversarial Corruption in Online Decision Problems
• Robust Stochastic Bandit Algorithms under Probabilistic Unbounded Adversarial Attack
• Combinatorial Bandits under Strategic Manipulations
• Accumulative Poisoning Attacks on Real-time Data
• Robust Bandit Learning with Imperfect Context
• Best-of-All-Worlds Bounds for Online Learning with Feedback Graphs