2 citations · 3 across the 9 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Approximate Muon with low-rank adapters
Ben Anson, Conor Houghton, Edward Milsom
The Muon optimizer shows clear benefits versus alternatives when pretraining neural networks. However, it is used less frequently for parameter-efficient fine-tuning (PEFT). One po…
cs.LG2025
Controlling changes to attention logits
Ben Anson, Laurence Aitchison
Stability of neural network weights is critical when training transformer models. The query and key weights are particularly problematic, as they tend to grow large without any int…