OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers

Siyuan Li, Jiabao Pan, Yumou Liu +9

Optimizer selection for large-scale model training has become a system-level design decision constrained jointly by compute, memory, tuning budget, and task diversity, yet the land…