Lion Secretly Solves Constrained Optimization: As Lyapunov Predicts

Lizhang Chen, Bo Liu, Kaizhao Liang +1

Lion (Evolved Sign Momentum), a new optimizer discovered through program search, has shown promising results in training large AI models. It performs comparably or favorably to Ada…