23 citations · 82 across the 9 of their papers we have counts for
9 papers
Generalized Leverage Score Sampling for Neural Networks
Jason D. Lee, Ruoqi Shen, Zhao Song +2
Leverage score sampling is a powerful technique that originates from theoretical computer science, which can be used to speed up a large number of fundamental questions, e.g. linea…
Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy
Edward Moroshko, Suriya Gunasekar, Blake Woodworth +3
We provide a detailed asymptotic study of gradient flow trajectories and their implicit optimization bias when minimizing the exponential loss over "diagonal linear networks". This…
Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks
Cong Fang, Jason D. Lee, Pengkun Yang +1
This paper proposes a new mean-field framework for over-parameterized deep neural networks (DNNs), which can be used to analyze neural network training. In this framework, a DNN is…
Shape Matters: Understanding the Implicit Bias of the Noise Covariance
Jeff Z. HaoChen, Colin Wei, Jason D. Lee +1
The noise in stochastic gradient descent (SGD) provides a crucial implicit regularization effect for training overparameterized models. Prior theoretical work largely focuses on sp…
Agnostic Q-learning with Function Approximation in Deterministic Systems: Tight Bounds on Approximation Error and Sample Complexity
Simon S. Du, Jason D. Lee, Gaurav Mahajan +1
The current paper studies the problem of agnostic -learning with function approximation in deterministic systems where the optimal -function is approximable by a function in…
When Does Non-Orthogonal Tensor Decomposition Have No Spurious Local Minima?
Maziar Sanjabi, Sina Baharlouei, Meisam Razaviyayn +1
We study the optimization problem for decomposing dimensional fourth-order Tensors with non-orthogonal components. We derive \textit{deterministic} conditions under which s…