Semiparametric Wavelet-based JPEG IV Estimator for endogenously truncated data

Abstract

A new and an enriched JPEG algorithm is provided for identifying redundancies in a sequence of irregular noisy data points which also accommodates a reference-free criterion function. Our main contribution is by formulating analytically (instead of approximating) the inverse of the transpose of JPEGwavelet transform without involving matrices which are computationally cumbersome. The algorithm is suitable for the widely-spread situations where the original data distribution is unobservable such as in cases where there is deficient representation of the entire population in the training data (in machine learning) and thus the covariate shift assumption is violated. The proposed estimator corrects for both biases, the one generated by endogenous truncation and the one generated by endogenous covariates. Results from utilizing 2,000,000 different distribution functions verify the applicability and high accuracy of our procedure to cases in which the disturbances are neither jointly nor marginally normally distributed.

arXiv comment18 pages

References in corpus (8)

• Nearly unbiased variable selection under minimax concave penalty
• Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
• Multivariate Archimedean copulas, $d$-monotone functions and ${\ell }_1$-norm symmetric distributions
• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
• Empirical Bayes selection of wavelet thresholds
• Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference
• Compression Approaches for the Regularized Solutions of Linear Systems from Large-Scale Inverse Problems
• Semiparametric correction for endogenous truncation bias with Vox Populi based participation decision