Pareto Optimal Model Selection in Linear Bandits

Abstract

We study model selection in linear bandits, where the learner must adapt to the dimension (denoted by $d_{\star }$) of the smallest hypothesis class containing the true linear model while balancing exploration and exploitation. Previous papers provide various guarantees for this model selection problem, but have limitations; i.e., the analysis requires favorable conditions that allow for inexpensive statistical testing to locate the right hypothesis class or are based on the idea of "corralling" multiple base algorithms, which often performs relatively poorly in practice. These works also mainly focus on upper bounds. In this paper, we establish the first lower bound for the model selection problem. Our lower bound implies that, even with a fixed action set, adaptation to the unknown dimension $d_{\star }$ comes at a cost: There is no algorithm that can achieve the regret bound $\tilde{O}(\sqrt{d_{\star }T})$ simultaneously for all values of $d_{\star }$. We propose Pareto optimal algorithms that match the lower bound. Empirical evaluations show that our algorithm enjoys superior performance compared to existing ones.

References in corpus (13)

• Black-box optimization of noisy functions with unknown smoothness
• Sequential Batch Learning in Finite-Action Linear Contextual Bandits
• Simultaneous Model Selection and Optimization through Parameter-free Stochastic Learning
• Model Selection in Contextual Stochastic Bandit Problems
• Adapting to Misspecification in Contextual Bandits
• High-Dimensional Sparse Linear Bandits
• Regret Bound Balancing and Elimination for Model Selection in Bandits and RL
• Regret Balancing for Bandit and RL Model Selection
• Corralling Stochastic Bandit Algorithms
• On Regret with Multiple Best Arms
• Upper Confidence Bounds for Combining Stochastic Bandits
• Open Problem: Model Selection for Contextual Bandits
• Bandit Theory meets Compressed Sensing for high dimensional Stochastic Linear Bandit

Cited by in corpus (1)

• Near Instance Optimal Model Selection for Pure Exploration Linear Bandits