Convergence Analysis of Gradient Descent Algorithms with Proportional Updates

Igor Gitman, Deepak Dilipkumar, Ben Parr

The rise of deep learning in recent years has brought with it increasingly clever optimization methods to deal with complex, non-linear loss functions. These methods are often desi…