Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Sign-SGD via Parameter-Free Optimization
Daniil Medyakov, Sergey Stanko, Gleb Molodtsov +4
Large language models have achieved major advances across domains, yet training them remains extremely resource-intensive. We revisit Sign-SGD, which serves both as a memory-effici…
cs.LG2026
Zero-Order Optimization for LLM Fine-Tuning via Learnable Direction Sampling
Valery Parfenov, Grigoriy Evseev, Andrey Veprikov +3
Fine-tuning large pretrained language models (LLMs) is a cornerstone of modern NLP, yet its growing memory demands (driven by backpropagation and large optimizer States) limit depl…
cs.LG2025
Leveraging Coordinate Momentum in SignSGD and Muon: Memory-Optimized Zero-Order
Egor Petrov, Grigoriy Evseev, Aleksey Antonov +4
Fine-tuning Large Language Models (LLMs) is essential for adapting pre-trained models to downstream tasks. Yet traditional first-order optimizers such as Stochastic Gradient Descen…