2 citations · 2 across the 7 of their papers we have counts for
4 papers · 1 filter
How do simple rotations affect the implicit bias of Adam?
Adela DePavia, Vasileios Charisopoulos, Rebecca Willett
Adaptive gradient methods such as Adam and Adagrad are widely used in machine learning, yet their effect on the generalization of learned models -- relative to methods like gradien…
Solving Inverse Problems with Deep Linear Neural Networks: Global Convergence Guarantees for Gradient Descent with Weight Decay
Hannah Laus, Suzanna Parkinson, Vasileios Charisopoulos +2
Machine learning methods are commonly used to solve inverse problems, wherein an unknown signal must be estimated from few indirect measurements generated via a known acquisition p…
Faster Adaptive Optimization via Expected Gradient Outer Product Reparameterization
Adela DePavia, Jose Cruzado, Jiayou Liang +2
Adaptive optimization algorithms -- such as Adagrad, Adam, and their variants -- have found widespread use in machine learning, signal processing and many other settings. Several m…
Depth Separation in Norm-Bounded Infinite-Width Neural Networks
Suzanna Parkinson, Greg Ongie, Rebecca Willett +2
We study depth separation in infinite-width neural networks, where complexity is controlled by the overall squared -norm of the weights (sum of squares of all weights in th…