Convergence rate and averaging of nonlinear two-time-scale stochastic approximation algorithms
arXiv:math/0610329 · doi:10.1214/105051606000000448
Abstract
The first aim of this paper is to establish the weak convergence rate of nonlinear two-time-scale stochastic approximation algorithms. Its second aim is to introduce the averaging principle in the context of two-time-scale stochastic approximation algorithms. We first define the notion of asymptotic efficiency in this framework, then introduce the averaged two-time-scale stochastic approximation algorithm, and finally establish its weak convergence rate. We show, in particular, that both components of the averaged two-time-scale stochastic approximation algorithm simultaneously converge at the optimal rate .
Published at http://dx.doi.org/10.1214/105051606000000448 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org)
References in corpus (2)
Cited by in corpus (18)
- Online EM Algorithm for Latent Data Models
- Non-asymptotic Convergence Analysis of Two Time-scale (Natural) Actor-Critic Algorithms
- Finite Time Analysis of Linear Two-timescale Stochastic Approximation with Markovian Noise
- Learning for Robust Routing Based on Stochastic Game in Cognitive Radio Networks
- Optimal Transport Based Distributionally Robust Optimization: Structural Properties and Iterative Schemes
- Interacting reinforced stochastic processes: statistical inference based on the weighted empirical means
- Networks of reinforced stochastic processes: asymptotics for the empirical means
- A Tale of Two-Timescale Reinforcement Learning with the Tightest Finite-Time Bound
- Sample Complexity Bounds for Two Timescale Value-based Reinforcement Learning Algorithms
- Finite-Time Convergence Rates of Nonlinear Two-Time-Scale Stochastic Approximation under Markovian Noise
- Generalized Rescaled Polya urn and its statistical applications
- Asymptotic study of stochastic adaptive algorithm in non-convex landscape
- A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learning
- Asymptotic Error Analysis of Multilevel Stochastic Approximations for the Value-at-Risk and Expected Shortfall
- Asymptotic Theory for IV-Based Reinforcement Learning with Potential Endogeneity
- Non-Asymptotic Analysis for Two Time-scale TDC with General Smooth Function Approximation
- A kernel-based stochastic approximation framework for contextual optimization
- Central limit theorems for interacting innovation processes, related statistical tools and general results