Online Learning in Contextual Bandits using Gated Linear Networks

Abstract

We introduce a new and completely online contextual bandit algorithm called Gated Linear Contextual Bandits (GLCB). This algorithm is based on Gated Linear Networks (GLNs), a recently introduced deep learning architecture with properties well-suited to the online setting. Leveraging data-dependent gating properties of the GLN we are able to estimate prediction uncertainty with effectively zero algorithmic overhead. We empirically evaluate GLCB compared to 9 state-of-the-art algorithms that leverage deep neural networks, on a standard benchmark suite of discrete and continuous contextual bandit problems. GLCB obtains median first-place despite being the only online method, and we further support these results with a theoretical study of its convergence properties.

arXiv commentNeurIPS 2020

References in corpus (6)

• A Contextual-Bandit Approach to Personalized News Article Recommendation
• Weight Uncertainty in Neural Networks
• Scalable Bayesian Optimization Using Deep Neural Networks
• #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning
• A Variational Analysis of Stochastic Gradient Algorithms
• Online Learning with Gated Linear Networks

Cited by in corpus (3)

• Hedging using reinforcement learning: Contextual $k$-Armed Bandit versus $Q$-learning
• Gaussian Gated Linear Networks
• A Combinatorial Perspective on Transfer Learning