Approximation of solutions of parameter-dependent problems by residual neural networks
arXiv:2607.13574 · doi:10.1515/jncds-2025-0019
The paper introduces a simple training scheme for residual neural networks based on analytic activation functions and gradient flows, with convergence guaranteed by Lojasiewicz theory, and shows it can approximate solutions of parameter-dependent ordinary differential equations and inverse wave problems.
Abstract
We develop a convergent scheme to train neural networks involving analytic activation functions based on gradient flows. Convergence properties are guaranteed by Lojasiewicz theory. The main advantage of this approach is its simplicity of implementation. The coefficients of the network are approximated by solving a system of ordinary differential equations. We test the method by constructing residual neural network approximations of solutions of parametric problems. The dependence of the solutions of simple ordinary differential equations on a few parameters is correctly reproduced. The solutions of inverse problems involving wave constraints which depend on a few parameters can be reasonably approximated, even in regions in which the problem is severely ill posed.
to appear in Journal of Nonlinear, Complex and Data Science