The Stochastic Reach-Avoid Problem and Set Characterization for Diffusions

Abstract

In this article we approach a class of stochastic reachability problems with state constraints from an optimal control perspective. Preceding approaches to solving these reachability problems are either confined to the deterministic setting or address almost-sure stochastic requirements. In contrast, we propose a methodology to tackle problems with less stringent requirements than almost sure. To this end, we first establish a connection between two distinct stochastic reach-avoid problems and three classes of stochastic optimal control problems involving discontinuous payoff functions. Subsequently, we focus on solutions of one of the classes of stochastic optimal control problems---the exit-time problem, which solves both the two reach-avoid problems mentioned above. We then derive a weak version of a dynamic programming principle (DPP) for the corresponding value function; in this direction our contribution compared to the existing literature is to develop techniques that admit discontinuous payoff functions. Moreover, based on our DPP, we provide an alternative characterization of the value function as a solution of a partial differential equation in the sense of discontinuous viscosity solutions, along with boundary conditions both in Dirichlet and viscosity senses. Theoretical justifications are also discussed to pave the way for deployment of off-the-shelf PDE solvers for numerical computations. Finally, we validate the performance of the proposed framework on the stochastic Zermelo navigation problem.

References in corpus (2)

• Controlled diffusion processes
• Maximizing the probability of attaining a target prior to extinction

Cited by in corpus (9)

• A dynamic game approach to distributionally robust safety specifications for stochastic systems
• A neural network based policy iteration algorithm with global $H^2$-superlinear convergence for stochastic games on domains
• Motion Planning for Continuous Time Stochastic Processes: A Dynamic Programming Approach
• Backward Reachability Approach to State-Constrained Stochastic Optimal Control Problems for Jump Diffusion Systems
• Physics-informed RL for Maximal Safety Probability Estimation
• On reachability of Markov chains: A long-run average approach
• Safety-Critical Control of Stochastic Systems using Stochastic Control Barrier Functions
• On the dynamic programming principle for controlled diffusion processes in a cylindrical region
• Convex programming in optimal control and information theory