Parametric estimation. Finite sample theory
arXiv:1111.3029 · doi:10.1214/12-AOS1054
Abstract
The paper aims at reconsidering the famous Le Cam LAN theory. The main features of the approach which make it different from the classical one are as follows: (1) the study is nonasymptotic, that is, the sample size is fixed and does not tend to infinity; (2) the parametric assumption is possibly misspecified and the underlying data distribution can lie beyond the given parametric family. These two features enable to bridge the gap between parametric and nonparametric theory and to build a unified framework for statistical estimation. The main results include large deviation bounds for the (quasi) maximum likelihood and the local quadratic bracketing of the log-likelihood process. The latter yields a number of important corollaries for statistical inference: concentration, confidence and risk bounds, expansion of the maximum likelihood estimate, etc. All these corollaries are stated in a nonclassical way admitting a model misspecification and finite samples. However, the classical asymptotic results including the efficiency bounds can be easily derived as corollaries of the obtained nonasymptotic statements. At the same time, the new bracketing device works well in the situations with large or growing parameter dimension in which the classical parametric theory fails. The general results are illustrated for the i.i.d. setup as well as for generalized linear and median estimation. The results apply for any dimension of the parameter space and provide a quantitative lower bound on the sample size yielding the root-n accuracy.
Published in at http://dx.doi.org/10.1214/12-AOS1054 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)
References in corpus (1)
Cited by in corpus (35)
- Bootstrap confidence sets under model misspecification
- Finite Sample Bernstein -- von Mises Theorem for Semiparametric Problems
- Bernstein - von Mises Theorem for growing parameter dimension
- Minimal penalties and the slope heuristics: a survey
- Recent theoretical advances in decentralized distributed convex optimization
- The Bernstein-von Mises theorem and nonregular models
- Derivative-Free Method For Composite Optimization With Applications To Decentralized Distributed Optimization
- Quantum metrology in the finite-sample regime
- Bayesian Sparse Linear Regression with Unknown Symmetric Error
- Stochastic gradient methods with inexact oracle
- High-Dimensional Composite Quantile Regression: Optimal Statistical Guarantees and Fast Algorithms
- Stochastic Optimization with Heavy-Tailed Noise via Accelerated Gradient Clipping
- Non-asymptotic theory for nonparametric testing
- Statistical Inference in Mean-Field Variational Bayes
- On Statistical Efficiency in Learning
- Accuracy of Gaussian approximation in nonparametric Bernstein -- von Mises Theorem
- High-Dimensional Quantile Regression: Convolution Smoothing and Concave Regularization
- Finite sample analysis of profile M-estimation in the Single Index model
- Evolutionary interpretations of entropy model for correspondence matrix calculation
- Bootstrap tuning in ordered model selection
- Bernstein - von Mises theorem and misspecified models: a review
- Inverse Reinforcement Learning from a Gradient-based Learner
- On the three-stage version of stable dynamic model
- Stochastic Approximation versus Sample Average Approximation for population Wasserstein barycenters
- Bayesian inference for nonlinear inverse problems
- Two convergence results for an alternation maximization procedure
- Parallel and Distributed algorithms for ML problems
- Traffic assignment models. Numerical aspects
- On rank estimators in increasing dimensions
- A result on the bias of sieve profile estimators
- Towards Model Selection for Local Log-Density Estimation. Fisher and Wilks-type theorems
- On MMSE estimation from quantized observations in the nonasymptotic regime
- A note on critical dimensions in profile semiparametric estimation
- Robust Inference via Multiplier Bootstrap
- Non-Asymptotic Behavior of the Maximum Likelihood Estimate of a Discrete Distribution