2 citations · 3 across the 8 of their papers we have counts for
8 papers
Approximate Muon with low-rank adapters
Ben Anson, Conor Houghton, Edward Milsom
The Muon optimizer shows clear benefits versus alternatives when pretraining neural networks. However, it is used less frequently for parameter-efficient fine-tuning (PEFT). One po…
Sharpness-Aware Minimization and Muon: Robustness under the Spectral Norm
Wenzhi Zhong, Edward Milsom, Michael Murray
Sharpness-Aware Minimization (SAM) aims to improve generalization by encouraging insensitivity to small, worst-case parameter perturbations. However, the notion of a "small" pertur…
Function-Space Learning Rates
Edward Milsom, Ben Anson, Laurence Aitchison
We consider layerwise function-space learning rates, which measure the magnitude of the change in a neural network's output function in response to an update to a parameter tensor.…
Stochastic Kernel Regularisation Improves Generalisation in Deep Kernel Machines
Edward Milsom, Ben Anson, Laurence Aitchison
Recent work developed convolutional deep kernel machines, achieving 92.7% test accuracy on CIFAR-10 using a ResNet-inspired architecture, which is SOTA for kernel methods. However,…
Flexible Infinite-Width Graph Convolutional Neural Networks
Ben Anson, Edward Milsom, Laurence Aitchison
A common theoretical approach to understanding neural networks is to take an infinite-width limit, at which point the outputs become Gaussian process (GP) distributed. This is know…
Convolutional Deep Kernel Machines
Edward Milsom, Ben Anson, Laurence Aitchison
Standard infinite-width limits of neural networks sacrifice the ability for intermediate layers to learn representations from data. Recent work (A theory of representation learning…
An Improved Variational Approximate Posterior for the Deep Wishart Process
Sebastian Ober, Ben Anson, Edward Milsom +1
Deep kernel processes are a recently introduced class of deep Bayesian models that have the flexibility of neural networks, but work entirely with Gram matrices. They operate by al…
A theory of representation learning gives a deep generalisation of kernel methods
Adam X. Yang, Maxime Robeyns, Edward Milsom +3
The successes of modern deep machine learning methods are founded on their ability to transform inputs across multiple layers to build good high-level representations. It is theref…