Approximation in shift-invariant spaces with deep ReLU neural networks

Abstract

We study the expressive power of deep ReLU neural networks for approximating functions in dilated shift-invariant spaces, which are widely used in signal processing, image processing, communications and so on. Approximation error bounds are estimated with respect to the width and depth of neural networks. The network construction is based on the bit extraction and data-fitting capacity of deep neural networks. As applications of our main results, the approximation rates of classical function spaces such as Sobolev spaces and Besov spaces are obtained. We also give lower bounds of the $L^p(1\le p\le \infty )$ approximation error for Sobolev spaces, which show that our construction of neural network is asymptotically optimal up to a logarithmic factor.

References in corpus (3)

• Deep Learning for Audio Signal Processing
• Deep Network Approximation for Smooth Functions
• Memory capacity of neural networks with threshold and ReLU activations

Cited by in corpus (8)

• Neural Network Approximation: Three Hidden Layers Are Enough
• Deep Network with Approximation Error Being Reciprocal of Width to Power of Square Root of Depth
• Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory
• Optimal rates of approximation by shallow ReLU${}^k$ neural networks and applications to nonparametric regression
• Approximation bounds for norm constrained neural networks with applications to regression and GANs
• Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neurons
• Solving PDEs on Unknown Manifolds with Machine Learning
• On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networks