Bounding the Difference between the Values of Robust and Non-Robust Markov Decision Problems

Abstract

In this note we provide an upper bound for the difference between the value function of a distributionally robust Markov decision problem and the value function of a non-robust Markov decision problem, where the ambiguity set of probability kernels of the distributionally robust Markov decision process is described by a Wasserstein-ball around some reference kernel whereas the non-robust Markov decision process behaves according to a fixed probability kernel contained in the ambiguity set. Our derived upper bound for the difference between the value functions is dimension-free and depends linearly on the radius of the Wasserstein-ball.

References in corpus (6)

• Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes
• Deep Hedging of Derivatives Using Reinforcement Learning
• Policy Gradient Method For Robust Reinforcement Learning
• Sample Complexity of Robust Reinforcement Learning with a Generative Model
• Policy Gradient Algorithms for Robust MDPs with Non-Rectangular Uncertainty Sets
• Non-concave stochastic optimal control in finite discrete time under model uncertainty