Approximation capability of two hidden layer feedforward neural networks with fixed weights

Abstract

We algorithmically construct a two hidden layer feedforward neural network (TLFN) model with the weights fixed as the unit coordinate vectors of the $d$-dimensional Euclidean space and having $3d+2$ number of hidden neurons in total, which can approximate any continuous $d$-variable function with an arbitrary precision. This result, in particular, shows an advantage of the TLFN model over the single hidden layer feedforward neural network (SLFN) model, since SLFNs with fixed weights do not have the capability of approximating multivariate functions.

13 pages, 3 figures; this article uses the algorithm from arXiv:1708.06219; for associated SageMath worksheet, see https://sites.google.com/site/njguliyev/papers/tlfn

References in corpus (3)

• On the approximation by single hidden layer feedforward neural networks with fixed weights
• A single hidden layer feedforward network with only one neuron in the hidden layer can approximate any univariate function
• Measure theoretic results for approximation by neural networks with limited weights

Cited by in corpus (11)

• Neural Network Approximation: Three Hidden Layers Are Enough
• SelectNet: Self-paced Learning for High-dimensional Partial Differential Equations
• Deep Network with Approximation Error Being Reciprocal of Width to Power of Square Root of Depth
• Convergence in Orlicz spaces by means of the multivariate max-product neural network operators of the Kantorovich type and applications
• Full error analysis for the training of deep neural networks
• Efficient approximation of high-dimensional functions with neural networks
• Are Two Hidden Layers Still Enough for the Physics-Informed Neural Networks?
• On the approximation of functions by tanh neural networks
• Negative results for approximation using single layer and multilayer feedforward neural networks
• Addressing common misinterpretations of KART and UAT in neural network literature
• On shallow feedforward neural networks with inputs from a topological space