Risk-Averse Multi-Armed Bandit Problems under Mean-Variance Measure

Abstract

The multi-armed bandit problems have been studied mainly under the measure of expected total reward accrued over a horizon of length $T$. In this paper, we address the issue of risk in multi-armed bandit problems and develop parallel results under the measure of mean-variance, a commonly adopted risk measure in economics and mathematical finance. We show that the model-specific regret and the model-independent regret in terms of the mean-variance of the reward process are lower bounded by $\Omega (\log T)$ and $\Omega (T^{2/3})$, respectively. We then show that variations of the UCB policy and the DSEE policy developed for the classic risk-neutral MAB achieve these lower bounds.

References in corpus (5)

• Risk-Aversion in Multi-armed Bandits
• Exploration vs Exploitation vs Safety: Risk-averse Multi-Armed Bandits
• Bounded regret in stochastic multi-armed bandits
• Generalized Risk-Aversion in Stochastic Multi-Armed Bandits
• Functional Bandits

Cited by in corpus (14)

• Robo-advising: Learning Investors' Risk Preferences via Portfolio Choices
• Secure Mobile Edge Computing in IoT via Collaborative Online Learning
• Thompson Sampling Algorithms for Mean-Variance Bandits
• Hedging using reinforcement learning: Contextual $k$-Armed Bandit versus $Q$-learning
• Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regret
• Risk-Constrained Thompson Sampling for CVaR Bandits
• Task Recommendation in Crowdsourcing Based on Learning Preferences and Reliabilities
• The Fragility of Optimized Bandit Algorithms
• Functional Sequential Treatment Allocation with Covariates
• The Survival Bandit Problem
• Decision Variance in Online Learning
• Rarely-switching linear bandits: optimization of causal effects for the real world
• Continuous Mean-Covariance Bandits
• Thompson Sampling for Gaussian Entropic Risk Bandits