Automatic Differentiation of Optimization Algorithms with Time-Varying Updates

Sheheryar Mehmood, Peter Ochs

Numerous Optimization Algorithms have a time-varying update rule thanks to, for instance, a changing step size, momentum parameter or, Hessian approximation. In this paper, we appl…