Markov Decision Processes with Recursive Risk Measures
arXiv:2010.07220 · doi:10.1016/j.ejor.2021.04.030
Abstract
In this paper, we consider risk-sensitive Markov Decision Processes (MDPs) with Borel state and action spaces and unbounded cost under both finite and infinite planning horizons. Our optimality criterion is based on the recursive application of static risk measures. This is motivated by recursive utilities in the economic literature, has been studied before for the entropic risk measure and is extended here to an axiomatic characterization of suitable risk measures. We derive a Bellman equation and prove the existence of Markovian optimal policies. For an infinite planning horizon, the model is shown to be contractive and the optimal policy to be stationary. Moreover, we establish a connection to distributionally robust MDPs, which provides a global interpretation of the recursively defined objective function. Monotone models are studied in particular.
References in corpus (6)
- Risk-sensitive Markov control processes
- Stochastic Optimal Growth Model with Risk Sensitive Preferences
- Distributionally Robust Markov Decision Processes and their Connection to Risk Measures
- A unified approach to time consistency of dynamic risk measures and dynamic performance measures in discrete time
- Practicality of Nested Risk Measures for Dynamic Electric Vehicle Charging
- Optimal Risk Allocation in Reinsurance Networks