Wasserstein Distributionally Robust Stochastic Control: A Data-Driven Approach

Abstract

Standard stochastic control methods assume that the probability distribution of uncertain variables is available. Unfortunately, in practice, obtaining accurate distribution information is a challenging task. To resolve this issue, we investigate the problem of designing a control policy that is robust against errors in the empirical distribution obtained from data. This problem can be formulated as a two-player zero-sum dynamic game problem, where the action space of the adversarial player is a Wasserstein ball centered at the empirical distribution. We propose computationally tractable value and policy iteration algorithms with explicit estimates of the number of iterations required for constructing an $\epsilon$-optimal policy. We show that the contraction property of associated Bellman operators extends a single-stage out-of-sample performance guarantee, obtained using a measure concentration inequality, to the corresponding multi-stage guarantee without any degradation in the confidence level. In addition, we characterize an explicit form of the optimal distributionally robust control policy and the worst-case distribution policy for linear-quadratic problems with Wasserstein penalty. Our study indicates that dynamic programming and Kantorovich duality play a critical role in solving and analyzing the Wasserstein distributionally robust stochastic control problems.

Cited by in corpus (16)

• Distributionally Robust Optimization: A Review
• Stochastic MPC with Distributionally Robust Chance Constraints
• Chance-Constrained Set Covering with Wasserstein Ambiguity
• Distributional Robustness and Regularization in Reinforcement Learning
• Wasserstein Distributionally Robust Motion Control for Collision Avoidance Using Conditional Value-at-Risk
• Robust Reinforcement Learning with Wasserstein Constraint
• Confidence Regions in Wasserstein Distributionally Robust Estimation
• Sinkhorn Distributionally Robust Optimization
• Safe Learning MPC with Limited Model Knowledge and Data
• Learning High Dimensional Wasserstein Geodesics
• Distributionally robust risk map for learning-based motion planning and control: A semidefinite programming approach
• Learning-based distributionally robust motion control with Gaussian processes
• Safe Wasserstein Constrained Deep Q-Learning
• Minimax control of ambiguous linear stochastic systems using the Wasserstein metric
• Data-driven Predictive Control for a Class of Uncertain Control-Affine Systems
• Discrete Distributionally Robust Optimal Control with Explicitly Constrained Optimization