A Note on Stability in Asynchronous Stochastic Approximation without Communication Delays

Abstract

In this paper, we study asynchronous stochastic approximation algorithms without communication delays. Our main contribution is a stability proof for these algorithms that extends a method of Borkar and Meyn by accommodating more general noise conditions. We also derive convergence results from this stability result and discuss their application in important average-reward reinforcement learning problems.

Corrected typos and a minor error; parts of this material will be included in a separate future arXiv preprint