activity
20242026
most citedAn Improved Empirical Fisher Approximation for Natural Gradient Descent

1 citations · 1 across the 4 of their papers we have counts for

collaborators
Showing 2026Show all

1 paper · 1 filter

cs.LG2026

Spectral Allocation: Why Muon Outperforms Adam, and How to Improve Muon

Xiaodong Wu, Wenyi Yu, Chao Zhang +1

Orthogonal optimisers such as Muon can substantially accelerate large language model pretraining relative to Adam, yet the mechanism remains incompletely understood. We investigate…