9 papers
When Does Preconditioning Help or Hurt Generalization?
Shun-ichi Amari, Jimmy Ba, Roger Grosse +5
While second order optimizers such as natural gradient descent (NGD) often speed up optimization, their effect on generalization has been called into question. This work presents a…
On the Optimal Weighted Regularization in Overparameterized Linear Regression
Denny Wu, Ji Xu
We consider the linear model with in the overparameterized regime . We estimate $\ma…
On the number of variables to use in principal component regression
Ji Xu, Daniel Hsu
We study least squares linear regression over uncorrelated Gaussian features that are selected in order of decreasing variance. When the number of selected features is at m…
Two models of double descent for weak features
Mikhail Belkin, Daniel Hsu, Ji Xu
The "double descent" risk curve was proposed to qualitatively describe the out-of-sample prediction accuracy of variably-parameterized machine learning models. This article provide…
Spectral Method for Phase Retrieval: an Expectation Propagation Perspective
Junjie Ma, Rishabh Dudeja, Ji Xu +2
Phase retrieval refers to the problem of recovering a signal from its phaseless measurements …
Consistent Risk Estimation in Moderately High-Dimensional Linear Regression
Ji Xu, Arian Maleki, Kamiar Rahnama Rad +1
Risk estimation is at the core of many learning systems. The importance of this problem has motivated researchers to propose different schemes, such as cross validation, generalize…