3 citations · 3 across the 9 of their papers we have counts for
9 papers
Learning Generation Orders for Masked Discrete Diffusion Models via Variational Inference
David Fox, Sam Bowyer, Song Liu +3
Masked discrete diffusion models (MDMs) are a promising new approach to generative modelling, offering the ability for parallel token generation and therefore greater efficiency th…
Controlling changes to attention logits
Ben Anson, Laurence Aitchison
Stability of neural network weights is critical when training transformer models. The query and key weights are particularly problematic, as they tend to grow large without any int…
Scale-invariant Attention
Ben Anson, Xi Wang, Laurence Aitchison
One persistent challenge in LLM research is the development of attention mechanisms that are able to generalise from training on shorter contexts to inference on longer contexts. W…
Function-Space Learning Rates
Edward Milsom, Ben Anson, Laurence Aitchison
We consider layerwise function-space learning rates, which measure the magnitude of the change in a neural network's output function in response to an update to a parameter tensor.…
Stochastic Kernel Regularisation Improves Generalisation in Deep Kernel Machines
Edward Milsom, Ben Anson, Laurence Aitchison
Recent work developed convolutional deep kernel machines, achieving 92.7% test accuracy on CIFAR-10 using a ResNet-inspired architecture, which is SOTA for kernel methods. However,…
Machine learning emulation of precipitation from km-scale UK regional climate simulations using a diffusion model
Henry Addison, Elizabeth Kendon, Suman Ravuri +2
High-resolution climate simulations are valuable for understanding climate change impacts. This has motivated use of regional convection-permitting climate models (CPMs), but these…
Questionable practices in machine learning
Gavin Leech, Juan J. Vazquez, Niclas Kupper +2
Evaluating modern ML models is hard. The strong incentive for researchers and companies to report a state-of-the-art result on some metric often leads to questionable research prac…
Flexible Infinite-Width Graph Convolutional Neural Networks
Ben Anson, Edward Milsom, Laurence Aitchison
A common theoretical approach to understanding neural networks is to take an infinite-width limit, at which point the outputs become Gaussian process (GP) distributed. This is know…
Convolutional Deep Kernel Machines
Edward Milsom, Ben Anson, Laurence Aitchison
Standard infinite-width limits of neural networks sacrifice the ability for intermediate layers to learn representations from data. Recent work (A theory of representation learning…