3 citations · 3 across the 9 of their papers we have counts for
Showing 2025Show all
3 papers · 1 filter
cs.LG2025
Controlling changes to attention logits
Ben Anson, Laurence Aitchison
Stability of neural network weights is critical when training transformer models. The query and key weights are particularly problematic, as they tend to grow large without any int…
cs.CL2025
Scale-invariant Attention
Ben Anson, Xi Wang, Laurence Aitchison
One persistent challenge in LLM research is the development of attention mechanisms that are able to generalise from training on shorter contexts to inference on longer contexts. W…
stat.ML2025
Function-Space Learning Rates
Edward Milsom, Ben Anson, Laurence Aitchison
We consider layerwise function-space learning rates, which measure the magnitude of the change in a neural network's output function in response to an update to a parameter tensor.…