Sign-SGD via Parameter-Free Optimization

Daniil Medyakov, Sergey Stanko, Gleb Molodtsov +4

Large language models have achieved major advances across domains, yet training them remains extremely resource-intensive. We revisit Sign-SGD, which serves both as a memory-effici…