Dynamic Game Theoretic Neural Optimizer

Abstract

The connection between training deep neural networks (DNNs) and optimal control theory (OCT) has attracted considerable attention as a principled tool of algorithmic design. Despite few attempts being made, they have been limited to architectures where the layer propagation resembles a Markovian dynamical system. This casts doubts on their flexibility to modern networks that heavily rely on non-Markovian dependencies between layers (e.g. skip connections in residual networks). In this work, we propose a novel dynamic game perspective by viewing each layer as a player in a dynamic game characterized by the DNN itself. Through this lens, different classes of optimizers can be seen as matching different types of Nash equilibria, depending on the implicit information structure of each (p)layer. The resulting method, called Dynamic Game Theoretic Neural Optimizer (DGNOpt), not only generalizes OCT-inspired optimizers to richer network class; it also motivates a new training principle by solving a multi-player cooperative game. DGNOpt shows convergence improvements over existing methods on image classification datasets with residual and inception networks. Our work marries strengths from both OCT and game theory, paving ways to new algorithmic opportunities from robust optimal control and bandit-based optimization.

arXiv commentAccepted in International Conference on Machine Learning (ICML) 2021 as Oral

References in corpus (13)

• Empirical Analysis of the Hessian of Over-Parametrized Neural Networks
• You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle
• Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis
• Neuron Shapley: Discovering the Responsible Neurons
• An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networks
• Deep Learning Theory Review: An Optimal Control and Dynamical Systems Perspective
• Analysing Neural Network Topologies: a Game Theoretic Approach
• Measurements of Three-Level Hierarchical Structure in the Outliers in the Spectrum of Deepnet Hessians
• A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depth
• Dissecting Hessian: Understanding Common Structure of Hessian in Neural Networks
• Mean-field Langevin System, Optimal Control and Deep Neural Networks
• Deep Online Convex Optimization with Gated Games
• DDPNOpt: Differential Dynamic Programming Neural Optimizer