Stochastic modified equations for the asynchronous stochastic gradient descent

Jing An, Jianfeng Lu, Lexing Ying

We propose a stochastic modified equations (SME) for modeling the asynchronous stochastic gradient descent (ASGD) algorithms. The resulting SME of Langevin type extracts more infor…