Randomized Allocation with Nonparametric Estimation for Contextual Multi-Armed Bandits with Delayed Rewards

Abstract

We study a multi-armed bandit problem with covariates in a setting where there is a possible delay in observing the rewards. Under some mild assumptions on the probability distributions for the delays and using an appropriate randomization to select the arms, the proposed strategy is shown to be strongly consistent.

arXiv commentAdded simulations and some minor typographical changes

References in corpus (7)

• A Contextual-Bandit Approach to Personalized News Article Recommendation
• Analysis of Thompson Sampling for the multi-armed bandit problem
• Contextual Bandits with Similarity Information
• Efficient Optimal Learning for Contextual Bandits
• Delay and Cooperation in Nonstochastic Bandits
• Bandits with Delayed, Aggregated Anonymous Feedback
• Stochastic Bandit Models for Delayed Conversions

Cited by in corpus (2)

• Metadata-based Multi-Task Bandits with Bayesian Hierarchical Models
• Nonstochastic Multiarmed Bandits with Unrestricted Delays