Risk averse non-stationary multi-armed bandits
arXiv:2109.13977
Abstract
This paper tackles the risk averse multi-armed bandits problem when incurred losses are non-stationary. The conditional value-at-risk (CVaR) is used as the objective function. Two estimation methods are proposed for this objective function in the presence of non-stationary losses, one relying on a weighted empirical distribution of losses and another on the dual representation of the CVaR. Such estimates can then be embedded into classic arm selection methods such as epsilon-greedy policies. Simulation experiments assess the performance of the arm selection algorithms based on the two novel estimation approaches, and such policies are shown to outperform naive benchmarks not taking non-stationarity into account.
References in corpus (7)
- A Survey on Practical Applications of Multi-Armed and Contextual Bandits
- Risk-Aversion in Multi-armed Bandits
- Exploration vs Exploitation vs Safety: Risk-averse Multi-Armed Bandits
- Optimal Thompson Sampling strategies for support-aware CVaR bandits
- Risk-Constrained Thompson Sampling for CVaR Bandits
- Bias-Corrected Peaks-Over-Threshold Estimation of the CVaR
- Risk-Averse Action Selection Using Extreme Value Theory Estimates of the CVaR