The Bias of Nonlinear Two-Time-scale Stochastic Approximation under Constant Step-Sizes

Djamel Rassem Lamouri, Dorian Baudry, Nicolas Gast

Two-timescale stochastic approximation (TTSA) is a fundamental tool for analyzing coupled iterative algorithms in reinforcement learning, optimization, and stochastic control. Howe…