Information Theory of Penalized Likelihoods and its Statistical Implications

Sabyasachi Chatterjee, Andrew Barron

We extend the correspondence between two-stage coding procedures in data compression and penalized likelihood procedures in statistical estimation. Traditionally, this had required…