Bias and Extrapolation in Markovian Linear Stochastic Approximation with Constant Stepsizes

Dongyan Huo, Yudong Chen, Qiaomin Xie

We consider Linear Stochastic Approximation (LSA) with a constant stepsize and Markovian data. Viewing the joint process of the data and LSA iterate as a time-homogeneous Markov ch…