Efficient approximation of high-dimensional functions with neural networks

Abstract

In this paper, we develop a framework for showing that neural networks can overcome the curse of dimensionality in different high-dimensional approximation problems. Our approach is based on the notion of a catalog network, which is a generalization of a standard neural network in which the nonlinear activation functions can vary from layer to layer as long as they are chosen from a predefined catalog of functions. As such, catalog networks constitute a rich family of continuous functions. We show that under appropriate conditions on the catalog, catalog networks can efficiently be approximated with rectified linear unit-type networks and provide precise estimates on the number of parameters needed for a given approximation accuracy. As special cases of the general results, we obtain different classes of functions that can be approximated with ReLU networks without the curse of dimensionality.

References in corpus (10)

• Why does deep and cheap learning work so well?
• Deep Network Approximation for Smooth Functions
• A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations
• On the approximation by single hidden layer feedforward neural networks with fixed weights
• Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networks
• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• Approximation capability of two hidden layer feedforward neural networks with fixed weights
• A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients
• On the ability of neural nets to express distributions
• Depth Separation for Neural Networks

Cited by in corpus (4)

• B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data
• Active learning of deep surrogates for PDEs: Application to metasurface design
• High-dimensional approximation spaces of artificial neural networks and applications to partial differential equations
• Fast approximate solvers for metamaterials design in electromagnetism