Infomax strategies for an optimal balance between exploration and exploitation
arXiv:1601.03073 · doi:10.1007/s10955-016-1521-0
Abstract
Proper balance between exploitation and exploration is what makes good decisions, which achieve high rewards like payoff or evolutionary fitness. The Infomax principle postulates that maximization of information directs the function of diverse systems, from living systems to artificial neural networks. While specific applications are successful, the validity of information as a proxy for reward remains unclear. Here, we consider the multi-armed bandit decision problem, which features arms (slot-machines) of unknown probabilities of success and a player trying to maximize cumulative payoff by choosing the sequence of arms to play. We show that an Infomax strategy (Info-p) which optimally gathers information on the highest mean reward among the arms saturates known optimal bounds and compares favorably to existing policies. The highest mean reward considered by Info-p is not the quantity actually needed for the choice of the arm to play, yet it allows for optimal tradeoffs between exploration and exploitation.
arXiv comment16 pages, 5 figures
References in corpus (7)
- ARACNE: An Algorithm for the Reconstruction of Gene Regulatory Networks in a Mammalian Cellular Context
- Rényi Divergence and Kullback-Leibler Divergence
- Information flow and optimization in transcriptional control
- Information transmission in genetic regulatory networks: a review
- The Value of Information for Populations in Varying Environments
- Kullback-Leibler upper confidence bounds for optimal sequential allocation
- Information theory and adaptation
Cited by in corpus (4)
- A high-bias, low-variance introduction to Machine Learning for physicists
- The End of Optimism? An Asymptotic Analysis of Finite-Armed Linear Bandits
- Weak transcription factor clustering at binding sites can facilitate information transfer from molecular signals
- Hunchback promoters can readout morphogenetic positional information in less than a minute