DDPNOpt: Differential Dynamic Programming Neural Optimizer

Abstract

Interpretation of Deep Neural Networks (DNNs) training as an optimal control problem with nonlinear dynamical systems has received considerable attention recently, yet the algorithmic development remains relatively limited. In this work, we make an attempt along this line by reformulating the training procedure from the trajectory optimization perspective. We first show that most widely-used algorithms for training DNNs can be linked to the Differential Dynamic Programming (DDP), a celebrated second-order method rooted in the Approximate Dynamic Programming. In this vein, we propose a new class of optimizer, DDP Neural Optimizer (DDPNOpt), for training feedforward and convolution networks. DDPNOpt features layer-wise feedback policies which improve convergence and reduce sensitivity to hyper-parameter over existing methods. It outperforms other optimal-control inspired training methods in both convergence and complexity, and is competitive against state-of-the-art first and second order methods. We also observe DDPNOpt has surprising benefit in preventing gradient vanishing. Our work opens up new avenues for principled algorithmic design built upon the optimal control theory.

arXiv commentAccepted in International Conference on Learning Representations (ICLR) 2021 as Spotlight

References in corpus (11)

• On the difficulty of training Recurrent Neural Networks
• Hamiltonian Neural Networks
• Optimizing Neural Networks with Kronecker-factored Approximate Curvature
• Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View
• Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
• Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis
• An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networks
• Deep Learning Theory Review: An Optimal Control and Dynamical Systems Perspective
• Practical Gauss-Newton Optimisation for Deep Learning
• Cross-Entropy Loss and Low-Rank Features Have Responsibility for Adversarial Examples
• Negative eigenvalues of the Hessian in deep neural networks

Cited by in corpus (3)

• Likelihood Training of Schrödinger Bridge using Forward-Backward SDEs Theory
• Second-Order Neural ODE Optimizer
• Dynamic Game Theoretic Neural Optimizer