Huan Li

activity

20172026

most citedAdan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models

63 citations · 137 across the 16 of their papers we have counts for

collaborators

Showing 2024Show all

2 papers · 1 filter

cs.LG2024

Convergence Rate Analysis of LION

Yiming Dong, Huan Li, Zhouchen Lin

The LION (evoLved sIgn mOmeNtum) optimizer for deep neural network training was found by Google via program search, with the simple sign update yet showing impressive performance i…

math.OC2024

On the $O (\frac{d}{T ^{1/4}})$ Convergence Rate of RMSProp and Its Momentum Extension Measured by $ℓ_{1}$ Norm

Huan Li, Yiming Dong, Zhouchen Lin

Although adaptive gradient methods have been extensively used in deep learning, their convergence rates proved in the literature are all slower than that of SGD, particularly with…

Convergence Rate Analysis of LION

On the O(T1/4d​​) Convergence Rate of RMSProp and Its Momentum Extension Measured by ℓ1​ Norm

On the $O (\frac{d}{T ^{1/4}})$ Convergence Rate of RMSProp and Its Momentum Extension Measured by $ℓ_{1}$ Norm