Gradient Descent Algorithm Survey

Deng Fucheng, Wang Wanjie, Gong Ao +2

Focusing on the practical configuration needs of optimization algorithms in deep learning, this article concentrates on five major algorithms: SGD, Mini-batch SGD, Momentum, Adam,…