Best-Arm Identification in Linear Bandits

Abstract

We study the best-arm identification problem in linear bandit, where the rewards of the arms depend linearly on an unknown parameter ${\theta }^{\ast }$ and the objective is to return the arm with the largest reward. We characterize the complexity of the problem and introduce sample allocation strategies that pull arms to identify the best arm with a fixed confidence, while minimizing the sample budget. In particular, we show the importance of exploiting the global linear structure to improve the estimate of the reward of near-optimal arms. We analyze the proposed strategies and compare their empirical performance. Finally, as a by-product of our analysis, we point out the connection to the $G$-optimality criterion used in optimal experimental design.

arXiv commentIn Advances in Neural Information Processing Systems 27 (NIPS), 2014

References in corpus (2)

• A Contextual-Bandit Approach to Personalized News Article Recommendation
• lil' UCB : An Optimal Exploration Algorithm for Multi-Armed Bandits

Cited by in corpus (33)

• Nearly Minimax-Optimal Regret for Linearly Parameterized Bandits
• An Empirical Process Approach to the Union Bound: Practical Algorithms for Combinatorial and Linear Bandits
• Optimal Best-arm Identification in Linear Bandits
• Polynomial-time Algorithms for Multiple-arm Identification with Full-bandit Feedback
• Asymptotically Optimal Information-Directed Sampling
• Sequential Experimental Design for Transductive Linear Bandits
• Explicit Best Arm Identification in Linear Bandits Using No-Regret Learners
• Towards Optimal and Efficient Best Arm Identification in Linear Bandits
• High-Dimensional Experimental Design and Kernel Bandits
• Design of Experiments for Stochastic Contextual Linear Bandits
• Sublinear Optimal Policy Value Estimation in Contextual Bandits
• Combinatorial Pure Exploration with Full-Bandit or Partial Linear Feedback
• Linear Bandits with Limited Adaptivity and Learning Distributional Optimal Design
• The Role of Contextual Information in Best Arm Identification
• Improved Algorithms for Agnostic Pool-based Active Classification
• Best Arm Identification with Safety Constraints
• Improved Confidence Bounds for the Linear Logistic Model and Applications to Linear Bandits
• Combinatorial Pure Exploration with Continuous and Separable Reward Functions and Its Applications (Extended Version)
• MaxGap Bandit: Adaptive Algorithms for Approximate Ranking
• Refined bounds for randomized experimental design
• Pure Exploration in Kernel and Neural Bandits
• Uniform-PAC Bounds for Reinforcement Learning with Linear Function Approximation
• Best Arm Identification in Graphical Bilinear Bandits
• Online Model Selection: a Rested Bandit Formulation
• Combinatorial Pure Exploration with Full-bandit Feedback and Beyond: Solving Combinatorial Optimization under Uncertainty with Limited Observation
• Diffusion Approximations for a Class of Sequential Testing Problems
• Learning to Actively Learn: A Robust Approach
• Collaborative Pure Exploration in Kernel Bandit
• Near Instance Optimal Model Selection for Pure Exploration Linear Bandits
• Thresholding Graph Bandits with GrAPL
• Price of Safety in Linear Best Arm Identification
• A Map of Bandits for E-commerce
• Chernoff Sampling for Active Testing and Extension to Active Regression