274 citations · 3.1k across the 75 of their papers we have counts for
7 papers · 1 filter
Plateau in Monotonic Linear Interpolation -- A "Biased" View of Loss Landscape for Deep Networks
Xiang Wang, Annie N. Wang, Mo Zhou +1
Monotonic linear interpolation (MLI) - on the line connecting a random initialization with the minimizer it converges to, the loss and accuracy are monotonic - is a phenomenon that…
Understanding Deflation Process in Over-parametrized Tensor Decomposition
Rong Ge, Yunwei Ren, Xiang Wang +1
In this paper we study the training dynamics for gradient flow on over-parametrized tensor decomposition problems. Empirically, such training process often first fits larger compon…
Beyond Lazy Training for Over-parameterized Tensor Decomposition
Xiang Wang, Chenwei Wu, Jason D. Lee +2
Over-parametrization is an important technique in training neural networks. In both theory and practice, training a larger network allows the optimization algorithm to avoid bad lo…
Guarantees for Tuning the Step Size using a Learning-to-Learn Approach
Xiang Wang, Shuai Yuan, Chenwei Wu +1
Choosing the right parameters for optimization algorithms is often the key to their success in practice. Solving this problem using a learning-to-learn approach -- using meta-gradi…
Homotopy Analysis for Tensor PCA
Anima Anandkumar, Yuan Deng, Rong Ge +1
Developing efficient and guaranteed nonconvex algorithms has been an important challenge in modern machine learning. Algorithms with good empirical performance such as stochastic g…
Un-regularizing: approximate proximal point and faster stochastic algorithms for empirical risk minimization
Roy Frostig, Rong Ge, Sham M. Kakade +1
We develop a family of accelerated stochastic algorithms that minimize sums of convex functions. Our algorithms improve upon the fastest running time for empirical risk minimizatio…