Showing stat.MLShow all
2 papers · 1 filter
stat.ML2026
Convergence of Muon with Newton-Schulz
Gyu Yeol Kim, Min-hwan Oh
We analyze Muon as originally proposed and used in practice -- using the momentum orthogonalization with a few Newton-Schulz steps. The prior theoretical results replace this key s…
stat.ML2025
ADAM Optimization with Adaptive Batch Selection
Gyu Yeol Kim, Min-hwan Oh
Adam is a widely used optimizer in neural network training due to its adaptive learning rate. However, because different data samples influence model updates to varying degrees, tr…