On the rate of convergence of an over-parametrized deep neural network regression estimate learned by gradient descent

Michael Kohler

paper

On the rate of convergence of an over-parametrized deep neural network regression estimate learned by gradient descent

arXiv:2504.03405

Abstract

Nonparametric regression with random design is considered. The $L_{2}$ error with integration with respect to the design measure is used as the error criterion. An over-parametrized deep neural network regression estimate with logistic activation function is defined, where all weights are learned by gradient descent. It is shown that the estimate achieves a nearly optimal rate of convergence in case that the regression function is $(p, C)$ --smooth.