Evaluation of Generalized Degrees of Freedom for Sparse Estimation by Replica Method
arXiv:1602.06506 · doi:10.1088/1742-5468/2016/12/123302
Abstract
We develop a method to evaluate the generalized degrees of freedom (GDF), which is a key quantity of a model selection criterion, for linear regression with sparse regularization. Using the replica method, GDF is expressed by the variables that characterize the saddle point of the free energy without depending on the form of the regularization. Within the framework of replica symmetric (RS) analysis, GDF is provided with a physical meaning as the effective density of non-zero components. The validity of our method in the RS phase is supported by the consistency of our results with previous mathematical results. The analytical results in the RS phase are numerically achieved by the belief propagation algorithm.
References in corpus (18)
- Discussion of "Least angle regression" by Efron et al
- Rejoinder to "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Least Angle Regression
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Message Passing Algorithms for Compressed Sensing
- Nonconcave penalized likelihood with a diverging number of parameters
- On the "degrees of freedom" of the lasso
- Statistical physics-based reconstruction in compressed sensing
- A typical reconstruction limit of compressed sensing based on Lp-norm minimization
- Cross validation in LASSO and its acceleration
- Statistical Mechanics of Dictionary Learning
- Sparse approximation based on a random overcomplete basis