Deep ReLU network approximation of functions on a manifold

Abstract

Whereas recovery of the manifold from data is a well-studied topic, approximation rates for functions defined on manifolds are less known. In this work, we study a regression problem with inputs on a $d^{\ast }$-dimensional manifold that is embedded into a space with potentially much larger ambient dimension. It is shown that sparsely connected deep ReLU networks can approximate a Hölder function with smoothness index $\beta$ up to error $\epsilon$ using of the order of ${\epsilon }^{-d^{\ast }/\beta }\log (1/\epsilon )$ many non-zero network parameters. As an application, we derive statistical convergence rates for the estimator minimizing the empirical risk over all possible choices of bounded network parameters.

References in corpus (4)

• Optimal approximation of continuous functions by very deep ReLU networks
• Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality
• On the minimax optimality and superiority of deep neural network learning over sparse parameter spaces
• Efficient Representation of Low-Dimensional Manifolds using Deep Networks

Cited by in corpus (15)

• The Modern Mathematics of Deep Learning
• Convergence Rate Analysis for Deep Ritz Method
• Black-Box Optimization with Local Generative Surrogates
• Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov space
• Nonlinear Causal Discovery with Confounders
• A likelihood approach to nonparametric estimation of a singular distribution using deep generative models
• Deep Nonparametric Regression on Approximate Manifolds: Non-Asymptotic Error Bounds with Polynomial Prefactors
• Deep Quantile Regression: Mitigating the Curse of Dimensionality Through Composition
• Classification Logit Two-sample Testing by Neural Networks
• Sharp Representation Theorems for ReLU Networks with Precise Dependence on Depth
• Robust Nonparametric Regression with Deep Neural Networks
• Non-asymptotic Excess Risk Bounds for Classification with Deep Convolutional Neural Networks
• Deep Networks Provably Classify Data on Curves
• Nearly Optimal Learning using Sparse Deep ReLU Networks in Regularized Empirical Risk Minimization with Lipschitz Loss
• Deep Regression for Repeated Measurements