Nonparametric Bandits with Covariates

Abstract

We consider a bandit problem which involves sequential sampling from two populations (arms). Each arm produces a noisy reward realization which depends on an observable random covariate. The goal is to maximize cumulative expected reward. We derive general lower bounds on the performance of any admissible policy, and develop an algorithm whose performance achieves the order of said lower bound up to logarithmic terms. This is done by decomposing the global problem into suitably "localized" bandit problems. Proofs blend ideas from nonparametric statistics and traditional methods used in the bandit literature.

References in corpus (1)

• Bandit Problems with Side Observations

Cited by in corpus (24)

• Contextual Bandits with Similarity Information
• Beyond UCB: Optimal and Efficient Contextual Bandits with Regression Oracles
• Sequential Batch Learning in Finite-Action Linear Contextual Bandits
• Instance-Dependent Complexity of Contextual Bandits and Reinforcement Learning: A Disagreement-Based Perspective
• Bounded regret in stochastic multi-armed bandits
• Nonparametric Pricing Analytics with Customer Covariates
• Optimal No-regret Learning in Repeated First-price Auctions
• Leveraging Good Representations in Linear Contextual Bandits
• Upper Counterfactual Confidence Bounds: a New Optimism Principle for Contextual Bandits
• Fast Rates for Contextual Linear Optimization
• Dynamic Batch Learning in High-Dimensional Sparse Linear Contextual Bandits
• Smooth Contextual Bandits: Bridging the Parametric and Non-differentiable Regret Regimes
• Nonparametric Stochastic Contextual Bandits
• Nonparametric Contextual Bandits in an Unknown Metric Space
• Bandits with an Edge
• Balanced Linear Contextual Bandits
• Regret Minimization in Isotonic, Heavy-Tailed Contextual Bandits via Adaptive Confidence Bands
• Tractable contextual bandits beyond realizability
• Self-Tuning Bandits over Unknown Covariate-Shifts
• Dimension Reduction in Contextual Online Learning via Nonparametric Variable Selection
• Functional Sequential Treatment Allocation with Covariates
• A Dimension-free Algorithm for Contextual Continuum-armed Bandits
• Sparse Reward Processes
• DTR Bandit: Learning to Make Response-Adaptive Decisions With Low Regret