Achieving Near Instance-Optimality and Minimax-Optimality in Stochastic and Adversarial Linear Bandits Simultaneously

Abstract

In this work, we develop linear bandit algorithms that automatically adapt to different environments. By plugging a novel loss estimator into the optimization problem that characterizes the instance-optimal strategy, our first algorithm not only achieves nearly instance-optimal regret in stochastic environments, but also works in corrupted environments with additional regret being the amount of corruption, while the state-of-the-art (Li et al., 2019) achieves neither instance-optimality nor the optimal dependence on the corruption amount. Moreover, by equipping this algorithm with an adversarial component and carefully-designed testings, our second algorithm additionally enjoys minimax-optimal regret in completely adversarial environments, which is the first of this kind to our knowledge. Finally, all our guarantees hold with high probability, while existing instance-optimal guarantees only hold in expectation.

References in corpus (13)

• Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits
• The best of both worlds: stochastic and adversarial bandits
• Efficient Optimal Learning for Contextual Bandits
• More Adaptive Algorithms for Adversarial Bandits
• An algorithm with nearly optimal pseudo-regret for both stochastic and adversarial bandits
• Beating Stochastic and Adversarial Semi-bandits Optimally and Simultaneously
• Stochastic Linear Optimization with Adversarial Corruption
• Corruption-robust exploration in episodic reinforcement learning
• Bias no more: high-probability data-dependent regret bounds for adversarial bandits and MDPs
• Regret Lower Bound and Optimal Algorithm in Finite Stochastic Partial Monitoring
• Taking a hint: How to leverage loss predictors in contextual bandits?
• Stochastic Linear Bandits Robust to Adversarial Attacks
• An Asymptotically Optimal Primal-Dual Incremental Algorithm for Contextual Linear Bandits

Cited by in corpus (5)

• A Model Selection Approach for Corruption Robust Reinforcement Learning
• Linear Contextual Bandits with Adversarial Corruptions
• On Optimal Robustness to Adversarial Corruption in Online Decision Problems
• Cooperative Stochastic Multi-agent Multi-armed Bandits Robust to Adversarial Corruptions
• Best-of-All-Worlds Bounds for Online Learning with Feedback Graphs