A Measure Theoretical Approach to the Mean-field Maximum Principle for Training NeurODEs

Abstract

In this paper we consider a measure-theoretical formulation of the training of NeurODEs in the form of a mean-field optimal control with $L^2$-regularization of the control. We derive first order optimality conditions for the NeurODE training problem in the form of a mean-field maximum principle, and show that it admits a unique control solution, which is Lipschitz continuous in time. As a consequence of this uniqueness property, the mean-field maximum principle also provides a strong quantitative generalization error for finite sample approximations. Our derivation of the mean-field maximum principle is much simpler than the ones currently available in the literature for mean-field optimal control problems, and is based on a generalized Lagrange multiplier theorem on convex sets of spaces of measures. The latter is also new, and can be considered as a result of independent interest.

References in corpus (7)

• A Mean Field View of the Landscape of Two-Layers Neural Networks
• A well-posedness theory in measures for some kinetic models of collective motion
• Recovery Guarantees for One-hidden-layer Neural Networks
• Mean-Field Sparse Optimal Control
• DNN Expression Rate Analysis of High-dimensional PDEs: Application to Option Pricing
• An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networks
• Differential Inclusions in Wasserstein Spaces: The Cauchy-Lipschitz Framework

Cited by in corpus (2)

• Deep Learning Approximation of Diffeomorphisms via Linear-Control Systems
• Semiconcavity and Sensitivity Analysis in Mean-Field Optimal Control and Applications