Stochastic Approximation and Newton's Estimate of a Mixing Distribution

Abstract

Many statistical problems involve mixture models and the need for computationally efficient methods to estimate the mixing distribution has increased dramatically in recent years. Newton [Sankhya Ser. A 64 (2002) 306--322] proposed a fast recursive algorithm for estimating the mixing distribution, which we study as a special case of stochastic approximation (SA). We begin with a review of SA, some recent statistical applications, and the theory necessary for analysis of a SA algorithm, which includes Lyapunov functions and ODE stability theory. Then standard SA results are used to prove consistency of Newton's estimate in the case of a finite mixture. We also propose a modification of Newton's algorithm that allows for estimation of an additional unknown parameter in the model, and prove its consistency.

Published in at http://dx.doi.org/10.1214/08-STS265 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (2)

• Equi-energy sampler with applications in statistical inference and statistical mechanics
• Consistency of a recursive estimate of mixing distributions

Cited by in corpus (15)

• Dempster--Shafer Theory and Statistical Inference with Weak Beliefs
• A nonparametric empirical Bayes framework for large-scale multiple testing
• On recursive Bayesian predictive distributions
• Consistency of a recursive estimate of mixing distributions
• Semiparametric inference in mixture models with predictive recursion marginal likelihood
• A semiparametric scale-mixture regression model and predictive recursion maximum likelihood
• Convergence rate for predictive recursion estimation of finite mixtures
• An approximate Bayesian marginal likelihood approach for estimating finite mixtures
• A class of measure-valued Markov chains and Bayesian nonparametrics
• On nonparametric estimation of a mixing density via the predictive recursion algorithm
• Fast nonparametric near-maximum likelihood estimation of a mixing density
• Revisiting consistency of a recursive estimator of mixing distributions
• Anytime valid and asymptotically optimal inference driven by predictive recursion
• Estimating a mixing distribution on the sphere using predictive recursion
• Computationally efficient variational-like approximations of possibilistic inferential models