112 citations · 289 across the 13 of their papers we have counts for
13 papers · 1 filter
Beyond Lazy Training for Over-parameterized Tensor Decomposition
Xiang Wang, Chenwei Wu, Jason D. Lee +2
Over-parametrization is an important technique in training neural networks. In both theory and practice, training a larger network allows the optimization algorithm to avoid bad lo…
Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot
Jingtong Su, Yihang Chen, Tianle Cai +4
Network pruning is a method for reducing test-time computational resource requirements with minimal performance degradation. Conventional wisdom of pruning algorithms suggests that…
How Important is the Train-Validation Split in Meta-Learning?
Yu Bai, Minshuo Chen, Pan Zhou +5
Meta-learning aims to perform fast adaptation on a new task through learning a "prior" from multiple existing tasks. A common practice in meta-learning is to perform a train-valida…
Generalized Leverage Score Sampling for Neural Networks
Jason D. Lee, Ruoqi Shen, Zhao Song +2
Leverage score sampling is a powerful technique that originates from theoretical computer science, which can be used to speed up a large number of fundamental questions, e.g. linea…
Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy
Edward Moroshko, Suriya Gunasekar, Blake Woodworth +3
We provide a detailed asymptotic study of gradient flow trajectories and their implicit optimization bias when minimizing the exponential loss over "diagonal linear networks". This…
Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks
Cong Fang, Jason D. Lee, Pengkun Yang +1
This paper proposes a new mean-field framework for over-parameterized deep neural networks (DNNs), which can be used to analyze neural network training. In this framework, a DNN is…