4 papers
Sharpness-Aware Minimization and Muon: Robustness under the Spectral Norm
Wenzhi Zhong, Edward Milsom, Michael Murray
Sharpness-Aware Minimization (SAM) aims to improve generalization by encouraging insensitivity to small, worst-case parameter perturbations. However, the notion of a "small" pertur…
Low Rank Gradients and Where to Find Them
Rishi Sonthalia, Michael Murray, Guido Montúfar
This paper investigates low-rank structure in the gradients of the training loss for two-layer neural networks while relaxing the usual isotropy assumptions on the training data an…
Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimension
Kedar Karhadkar, Michael Murray, Guido Montúfar
Bounds on the smallest eigenvalue of the neural tangent kernel (NTK) are a key ingredient in the analysis of neural network optimization and memorization. However, existing results…
Benign overfitting in leaky ReLU networks with moderate input dimension
Kedar Karhadkar, Erin George, Michael Murray +2
The problem of benign overfitting asks whether it is possible for a model to perfectly fit noisy training data and still generalize well. We study benign overfitting in two-layer l…