A Bit Better? Quantifying Information for Bandit Learning
arXiv:2102.09488
Abstract
The information ratio offers an approach to assessing the efficacy with which an agent balances between exploration and exploitation. Originally, this was defined to be the ratio between squared expected regret and the mutual information between the environment and action-observation pair, which represents a measure of information gain. Recent work has inspired consideration of alternative information measures, particularly for use in analysis of bandit learning algorithms to arrive at tighter regret bounds. We investigate whether quantification of information via such alternatives can improve the realized performance of information-directed sampling, which aims to minimize the information ratio.
arXiv comment41 pages, 10 figures, 1 table
References in corpus (7)
- Further Optimal Regret Bounds for Thompson Sampling
- An Information-Theoretic Approach to Minimax Regret in Partial Monitoring
- An Information-Theoretic Analysis for Thompson Sampling with Many Actions
- Information-Theoretic Confidence Bounds for Reinforcement Learning
- Asymptotically Optimal Information-Directed Sampling
- Connections Between Mirror Descent, Thompson Sampling and the Information Ratio
- Mirror Descent and the Information Ratio