Neural Contextual Bandits with UCB-based Exploration

Abstract

We study the stochastic contextual bandit problem, where the reward is generated from an unknown function with additive noise. No assumption is made about the reward function other than boundedness. We propose a new algorithm, NeuralUCB, which leverages the representation power of deep neural networks and uses a neural network-based random feature mapping to construct an upper confidence bound (UCB) of reward for efficient exploration. We prove that, under standard assumptions, NeuralUCB achieves $\tilde{O}(\sqrt{T})$ regret, where $T$ is the number of rounds. To the best of our knowledge, it is the first neural network-based contextual bandit algorithm with a near-optimal regret guarantee. We also show the algorithm is empirically competitive against representative baselines in a number of benchmarks.

arXiv comment27 pages, 2 figures, 1 table. In ICML 2020

Cited by in corpus (22)

• PairRank: Online Pairwise Learning to Rank by Divide-and-Conquer
• Neural Contextual Bandits with Deep Representation and Shallow Exploration
• Graph Neural Bandits
• EE-Net: Exploitation-Exploration Neural Networks in Contextual Bandits
• Ed-Fed: A generic federated learning framework with resource-aware client selection for edge devices
• Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvature
• Optimal Order Simple Regret for Gaussian Process Bandits
• Neural Bandit with Arm Group Graph
• Multi-facet Contextual Bandits: A Neural Network Perspective
• Risk-Aware Continuous Control with Neural Contextual Bandits
• Efficient Online Bayesian Inference for Neural Bandits
• Offline Neural Contextual Bandits: Pessimism, Optimization and Generalization
• Online Limited Memory Neural-Linear Bandits with Likelihood Matching
• CORe: Capitalizing On Rewards in Bandit Exploration
• Adversarial Linear Contextual Bandits with Graph-Structured Side Observations
• Quantifying Epistemic Uncertainty in Deep Learning
• Robust Generalization of Quadratic Neural Networks via Function Identification
• Syndicated Bandits: A Framework for Auto Tuning Hyper-parameters in Contextual Bandit Algorithms
• Regularized OFU: an Efficient UCB Estimator forNon-linear Contextual Bandit
• A Map of Bandits for E-commerce
• Uniform Generalization Bounds for Overparameterized Neural Networks
• The Adaptive Doubly Robust Estimator for Policy Evaluation in Adaptive Experiments and a Paradox Concerning Logging Policy