Gamification of Pure Exploration for Linear Bandits

Abstract

We investigate an active pure-exploration setting, that includes best-arm identification, in the context of linear stochastic bandits. While asymptotically optimal algorithms exist for standard multi-arm bandits, the existence of such algorithms for the best-arm identification in linear bandits has been elusive despite several attempts to address it. First, we provide a thorough comparison and new insight over different notions of optimality in the linear case, including G-optimality, transductive optimality from optimal experimental design and asymptotic optimality. Second, we design the first asymptotically optimal algorithm for fixed-confidence pure exploration in linear bandits. As a consequence, our algorithm naturally bypasses the pitfall caused by a simple but difficult instance, that most prior algorithms had to be engineered to deal with explicitly. Finally, we avoid the need to fully solve an optimal design problem by providing an approach that entails an efficient implementation.

arXiv comment11+25 pages. To be published in the proceedings of ICML 2020

Cited by in corpus (16)

• Leveraging Good Representations in Linear Contextual Bandits
• High-Dimensional Experimental Design and Kernel Bandits
• An Asymptotically Optimal Primal-Dual Incremental Algorithm for Contextual Linear Bandits
• Design of Experiments for Stochastic Contextual Linear Bandits
• Combinatorial Pure Exploration with Full-Bandit or Partial Linear Feedback
• Efficient Pure Exploration for Combinatorial Bandits with Semi-Bandit Feedback
• Improved Confidence Bounds for the Linear Logistic Model and Applications to Linear Bandits
• Pure Exploration in Kernel and Neural Bandits
• Price of Safety in Linear Best Arm Identification
• Combinatorial Pure Exploration with Full-bandit Feedback and Beyond: Solving Combinatorial Optimization under Uncertainty with Limited Observation
• Chernoff Sampling for Active Testing and Extension to Active Regression
• Stochastic Bandits with Vector Losses: Minimizing ${\ell }^{\infty }$-Norm of Relative Losses
• Learning to Actively Learn: A Robust Approach
• Near Instance Optimal Model Selection for Pure Exploration Linear Bandits
• Nearly Optimal Algorithms for Level Set Estimation
• Guaranteed Fixed-Confidence Best Arm Identification in Multi-Armed Bandits: Simple Sequential Elimination Algorithms