4 papers
Sharpness-Aware Minimization and Muon: Robustness under the Spectral Norm
Wenzhi Zhong, Edward Milsom, Michael Murray
Sharpness-Aware Minimization (SAM) aims to improve generalization by encouraging insensitivity to small, worst-case parameter perturbations. However, the notion of a "small" pertur…
Flexible Infinite-Width Graph Convolutional Neural Networks
Ben Anson, Edward Milsom, Laurence Aitchison
A common theoretical approach to understanding neural networks is to take an infinite-width limit, at which point the outputs become Gaussian process (GP) distributed. This is know…
Function-Space Learning Rates
Edward Milsom, Ben Anson, Laurence Aitchison
We consider layerwise function-space learning rates, which measure the magnitude of the change in a neural network's output function in response to an update to a parameter tensor.…
Stochastic Kernel Regularisation Improves Generalisation in Deep Kernel Machines
Edward Milsom, Ben Anson, Laurence Aitchison
Recent work developed convolutional deep kernel machines, achieving 92.7% test accuracy on CIFAR-10 using a ResNet-inspired architecture, which is SOTA for kernel methods. However,…