Regress-Later Monte Carlo for optimal control of Markov processes

Abstract

We develop two Regression Monte Carlo algorithms (value and performance iteration) to solve general problems of optimal stochastic control of discrete-time Markov processes. We formulate our method within an innovative framework that allow us to prove the speed of convergence of our numerical schemes. We rely on the Regress Later approach unlike other attempts which employ the Regress Now technique. We exploit error bounds obtained in our proofs, along with numerical experiments, to investigate differences between the value and performance iteration approaches. Introduced in Tsitsiklis and VanRoy [2001] and Longstaff and Schwartz [2001] respectively, their characteristics have gone largely unnoticed in the literature; we show however that their differences are paramount in practical solution of stochastic control problems. Finally, we provide some guidelines for the tuning of our algorithms.

arXiv comment31 pages

References in corpus (7)

• A regression-based Monte Carlo method to solve backward stochastic differential equations
• A numerical algorithm for fully nonlinear HJB equations: an approach by control randomization
• Simulation of BSDEs by Wiener chaos expansion
• A dynamic look-ahead Monte Carlo algorithm for pricing Bermudan options
• Regress-Later Monte Carlo for Optimal Inventory Control with applications in energy
• Fast Convergence of Regress-Later Estimates in Least Squares Monte Carlo
• A Regress-Later Algorithm for Backward Stochastic Differential Equations

Cited by in corpus (3)

• Statistical Learning for Probability-Constrained Stochastic Optimal Control
• Robust utility maximization under model uncertainty via a penalization approach
• Automatic model training under restrictive time constraints