Self-Supervised Contextual Bandits in Computer Vision
arXiv:2003.08485
Abstract
Contextual bandits are a common problem faced by machine learning practitioners in domains as diverse as hypothesis testing to product recommendations. There have been a lot of approaches in exploiting rich data representations for contextual bandit problems with varying degree of success. Self-supervised learning is a promising approach to find rich data representations without explicit labels. In a typical self-supervised learning scheme, the primary task is defined by the problem objective (e.g. clustering, classification, embedding generation etc.) and the secondary task is defined by the self-supervision objective (e.g. rotation prediction, words in neighborhood, colorization, etc.). In the usual self-supervision, we learn implicit labels from the training data for a secondary task. However, in the contextual bandit setting, we don't have the advantage of getting implicit labels due to lack of data in the initial phase of learning. We provide a novel approach to tackle this issue by combining a contextual bandit objective with a self supervision objective. By augmenting contextual bandit learning with self-supervision we get a better cumulative reward. Our results on eight popular computer vision datasets show substantial gains in cumulative reward. We provide cases where the proposed scheme doesn't perform optimally and give alternative methods for better learning in these cases.
References in corpus (11)
- Efficient Estimation of Word Representations in Vector Space
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Thompson Sampling for Contextual Bandits with Linear Payoffs
- Unbiased Offline Evaluation of Contextual-bandit-based News Article Recommendation Algorithms
- Large-Scale Study of Curiosity-Driven Learning
- A Theoretical Analysis of Contrastive Unsupervised Representation Learning
- Exploiting Unlabeled Data in CNNs by Self-supervised Learning to Rank
- Finite-Time Analysis of Kernelised Contextual Bandits
- Self-Supervised Exploration via Disagreement
- Simple Regret Minimization for Contextual Bandits
- Two Stream Self-Supervised Learning for Action Recognition