Markovian stochastic approximation with expanding projections

Abstract

Stochastic approximation is a framework unifying many random iterative algorithms occurring in a diverse range of applications. The stability of the process is often difficult to verify in practical applications and the process may even be unstable without additional stabilisation techniques. We study a stochastic approximation procedure with expanding projections similar to Andradóttir [Oper. Res. 43 (1995) 1037-1048]. We focus on Markovian noise and show the stability and convergence under general conditions. Our framework also incorporates the possibility to use a random step size sequence, which allows us to consider settings with a non-smooth family of Markov kernels. We apply the theory to stochastic approximation expectation maximisation with particle independent Metropolis-Hastings sampling.

Published in at http://dx.doi.org/10.3150/12-BEJ497 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm)

References in corpus (3)

• On the ergodicity properties of some adaptive MCMC algorithms
• On the ergodicity of the adaptive Metropolis algorithm on unbounded domains
• On the stability and ergodicity of adaptive scaling Metropolis algorithms

Cited by in corpus (4)

• On the efficiency of pseudo-marginal random walk Metropolis algorithms
• On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoods
• Convergence properties of pseudo-marginal Markov chain Monte Carlo algorithms
• On the stability of some controlled Markov chains and its applications to stochastic approximation with Markovian dynamic