Quantified advantage of discontinuous weight selection in approximations with deep neural networks

Abstract

We consider approximations of 1D Lipschitz functions by deep ReLU networks of a fixed width. We prove that without the assumption of continuous weight selection the uniform approximation error is lower than with this assumption at least by a factor logarithmic in the size of the network.

12 pages, submitted to J. Approx. Theory

Cited by in corpus (3)

• Nonlinear Approximation and (Deep) ReLU Networks
• Calibration of a Fluid-Structure Problem with Keras
• High-dimensional nonlinear approximation by parametric manifolds in Hölder-Nikol'skii spaces of mixed smoothness