Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
In Search of Adam's Secret Sauce
Antonio Orvieto, Robert M. Gower
Understanding the remarkable efficacy of Adam when training transformer-based language models has become a central research topic within the optimization community. To gain deeper…
cs.LG2025
The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm
Noah Amsel, David Persson, Christopher Musco +1
Computing the polar decomposition and the related matrix sign function has been a well-studied problem in numerical analysis for decades. Recently, it has emerged as an important s…