Statistical Inference with Local Optima
arXiv:1807.04431
Abstract
We study the statistical properties of an estimator derived by applying a gradient ascent method with multiple initializations to a multi-modal likelihood function. We derive the population quantity that is the target of this estimator and study the properties of confidence intervals (CIs) constructed from asymptotic normality and the bootstrap approach. In particular, we analyze the coverage deficiency due to finite number of random initializations. We also investigate the CIs by inverting the likelihood ratio test, the score test, and the Wald test, and we show that the resulting CIs may be very different. We propose a two-sample test procedure even when the MLE is intractable. In addition, we analyze the performance of the EM algorithm under random initializations and derive the coverage of a CI with a finite number of initializations.
To appear in the Journal of American Statistical Association. Main paper 31 pages, 6 figures
References in corpus (6)
- Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis
- Robust Topological Inference: Distance To a Measure and Kernel Distance
- The Landscape of Empirical Risk for Non-convex Losses
- Local Maxima in the Likelihood of Gaussian Mixture Models: Structural Results and Algorithmic Consequences
- Statistical Inference for the Population Landscape via Moment Adjusted Stochastic Gradients
- Risk Bounds For Mode Clustering