An Isometric Stochastic Optimizer

Jacob Jackson

The Adam optimizer is the standard choice in deep learning applications. I propose a simple explanation of Adam's success: it makes each parameter's step size independent of the no…