Risk-sensitive Markov control processes
arXiv:1110.6317 · doi:10.1137/120899005
Abstract
We introduce a general framework for measuring risk in the context of Markov control processes with risk maps on general Borel spaces that generalize known concepts of risk measures in mathematical finance, operations research and behavioral economics. Within the framework, applying weighted norm spaces to incorporate also unbounded costs, we study two types of infinite-horizon risk-sensitive criteria, discounted total risk and average risk, and solve the associated optimization problems by dynamic programming. For the discounted case, we propose a new discount scheme, which is different from the conventional form but consistent with the existing literature, while for the average risk criterion, we state Lyapunov-like stability conditions that generalize known conditions for Markov chains to ensure the existence of solutions to the optimality equation.
arXiv comment21 pages
Cited by in corpus (17)
- Risk-sensitive Reinforcement Learning
- Zero-sum Risk-Sensitive Stochastic Games
- Markov Decision Processes with Risk-Sensitive Criteria: An Overview
- Markov Decision Processes with Recursive Risk Measures
- Generalized Risk-Aversion in Stochastic Multi-Armed Bandits
- Ergodic Risk-Sensitive Control of Markov Processes on Countable State Space Revisited
- Distributionally Robust Markov Decision Processes and their Connection to Risk Measures
- Prospect-theoretic Q-learning
- Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regret
- Stochastic Approximation for Risk-aware Markov Decision Processes
- Process-Based Risk Measures and Risk-Averse Control of Discrete-Time Systems
- Discrete-time risk-aware optimal switching with non-adapted costs
- Off-Policy Risk-Sensitive Reinforcement Learning Based Constrained Robust Optimal Control
- Risk-Averse Learning by Temporal Difference Methods
- Model and Reinforcement Learning for Markov Games with Risk Preferences
- Index-Based Policy for Risk-Averse Multi-Armed Bandit
- On Average Risk-sensitive Markov Control Processes