Towards Understanding the Generalizability of Delayed Stochastic Gradient Descent

Xiaoge Deng, Li Shen, Shengwei Li +3

Stochastic gradient descent (SGD) performed in an asynchronous manner plays a crucial role in training large-scale machine learning models. However, the generalization performance…