Bias in parametric estimation: reduction and useful side-effects
arXiv:1311.6311 · doi:10.1002/wics.1296
Abstract
The bias of an estimator is defined as the difference of its expected value from the parameter to be estimated, where the expectation is with respect to the model. Loosely speaking, small bias reflects the desire that if an experiment is repeated indefinitely then the average of all the resultant estimates will be close to the parameter value that is estimated. The current paper is a review of the still-expanding repository of methods that have been developed to reduce bias in the estimation of parametric models. The review provides a unifying framework where all those methods are seen as attempts to approximate the solution of a simple estimating equation. Of particular focus is the maximum likelihood estimator, which despite being asymptotically unbiased under the usual regularity conditions, has finite-sample bias that can result in significant loss of performance of standard inferential procedures. An informal comparison of the methods is made revealing some useful practical side-effects in the estimation of popular models in practice including: i) shrinkage of the estimators in binomial and multinomial regression models that guarantees finiteness even in cases of data separation where the maximum likelihood estimator is infinite, and ii) inferential benefits for models that require the estimation of dispersion or precision parameters.
References in corpus (1)
Cited by in corpus (11)
- Bias-Reduced Neural Networks for Parameter Estimation in Quantitative MRI
- Biases in Inverse Ising Estimates of Near-Critical Behaviour
- Asymptotically Optimal Bias Reduction for Parametric Models
- Bias-Reduced Estimation of Structural Equation Models
- Tuning in ridge logistic regression to solve separation
- Phase Transition Unbiased Estimation in High Dimensional Settings
- Information-Corrected Estimation: A Generalization Error Reducing Parameter Estimation Method
- A General Approach for Simulation-based Bias Correction in High Dimensional Settings
- Accurate inference in negative binomial regression
- Bring the noise: exact inference from noisy simulations in collider physics
- A Class of Computational Methods to Reduce Selection Bias when Designing Phase 3 Clinical Trials