Design of Experiments for Stochastic Contextual Linear Bandits
arXiv:2107.09912
Abstract
In the stochastic linear contextual bandit setting there exist several minimax procedures for exploration with policies that are reactive to the data being acquired. In practice, there can be a significant engineering overhead to deploy these algorithms, especially when the dataset is collected in a distributed fashion or when a human in the loop is needed to implement a different policy. Exploring with a single non-reactive policy is beneficial in such cases. Assuming some batch contexts are available, we design a single stochastic policy to collect a good dataset from which a near-optimal policy can be extracted. We present a theoretical analysis as well as numerical experiments on both synthetic and real-world datasets.
arXiv commentInitial submission
References in corpus (9)
- Contextual Bandit Algorithms with Supervised Learning Guarantees
- Best-Arm Identification in Linear Bandits
- Sequential Batch Learning in Finite-Action Linear Contextual Bandits
- Gamification of Pure Exploration for Linear Bandits
- Simple Regret Minimization for Contextual Bandits
- Sequential Experimental Design for Transductive Linear Bandits
- Regret Bounds for Batched Bandits
- Cautiously Optimistic Policy Optimization and Exploration with Linear Function Approximation
- Provably Efficient Reinforcement Learning with Linear Function Approximation Under Adaptivity Constraints