Leveraging Coordinate Momentum in SignSGD and Muon: Memory-Optimized Zero-Order

Egor Petrov, Grigoriy Evseev, Aleksey Antonov +4

Fine-tuning Large Language Models (LLMs) is essential for adapting pre-trained models to downstream tasks. Yet traditional first-order optimizers such as Stochastic Gradient Descen…