1 citations · 2 across the 8 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.LG2023★ 1 cited
Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos
Dayal Singh Kalra, Tianyu He, Maissam Barkeshli
In gradient descent dynamics of neural networks, the top eigenvalue of the loss Hessian (sharpness) displays a variety of robust phenomena throughout training. This includes early…
cs.LG2023★ 1 cited
To grok or not to grok: Disentangling generalization and memorization on corrupted algorithmic datasets
Darshil Doshi, Aritra Das, Tianyu He +1
Robust generalization is a major challenge in deep learning, particularly when the number of trainable parameters is very large. In general, it is very difficult to know if the net…