Risk-Aversion in Multi-armed Bandits

Abstract

Stochastic multi-armed bandits solve the Exploration-Exploitation dilemma and ultimately maximize the expected reward. Nonetheless, in many practical problems, maximizing the expected reward is not the most desirable objective. In this paper, we introduce a novel setting based on the principle of risk-aversion where the objective is to compete against the arm with the best risk-return trade-off. This setting proves to be intrinsically more difficult than the standard multi-arm bandit setting due in part to an exploration risk which introduces a regret associated to the variability of an algorithm. Using variance as a measure of risk, we introduce two new algorithms, investigate their theoretical guarantees, and report preliminary empirical results.

(2012)

Cited by in corpus (27)

• Risk-Averse Multi-Armed Bandit Problems under Mean-Variance Measure
• Robo-advising: Learning Investors' Risk Preferences via Portfolio Choices
• What Doubling Tricks Can and Can't Do for Multi-Armed Bandits
• Exploration vs Exploitation vs Safety: Risk-averse Multi-Armed Bandits
• Distribution oblivious, risk-aware algorithms for multi-armed bandits with unbounded rewards
• Thompson Sampling Algorithms for Mean-Variance Bandits
• Hedging using reinforcement learning: Contextual $k$-Armed Bandit versus $Q$-learning
• Learning Bounds for Risk-sensitive Learning
• Adaptive Sampling for Stochastic Risk-Averse Learning
• Off-Policy Risk Assessment in Contextual Bandits
• X-Armed Bandits: Optimizing Quantiles, CVaR and Other Risks
• Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regret
• Optimal Best-Arm Identification Methods for Tail-Risk Measures
• Risk-Constrained Thompson Sampling for CVaR Bandits
• Asynchronous Parallel Empirical Variance Guided Algorithms for the Thresholding Bandit Problem
• Task Recommendation in Crowdsourcing Based on Learning Preferences and Reliabilities
• The Fragility of Optimized Bandit Algorithms
• Index-Based Policy for Risk-Averse Multi-Armed Bandit
• The Survival Bandit Problem
• Functional Sequential Treatment Allocation with Covariates
• Decision Variance in Online Learning
• A Reliability-aware Multi-armed Bandit Approach to Learn and Select Users in Demand Response
• Risk-Averse Action Selection Using Extreme Value Theory Estimates of the CVaR
• Continuous Mean-Covariance Bandits
• Concentration bounds for CVaR estimation: The cases of light-tailed and heavy-tailed distributions
• Thompson Sampling for Gaussian Entropic Risk Bandits
• Risk averse non-stationary multi-armed bandits