EAdam Optimizer: How $\epsilon$ Impact Adam

Wei Yuan, Kai-Xin Gao

Many adaptive optimization methods have been proposed and used in deep learning, in which Adam is regarded as the default algorithm and widely used in many deep learning frameworks…