AdaFamily: A family of Adam-like adaptive gradient methods

Hannes Fassold

We propose AdaFamily, a novel method for training deep neural networks. It is a family of adaptive gradient methods and can be interpreted as sort of a blend of the optimization al…