On the approximation by single hidden layer feedforward neural networks with fixed weights

Abstract

Feedforward neural networks have wide applicability in various disciplines of science due to their universal approximation property. Some authors have shown that single hidden layer feedforward neural networks (SLFNs) with fixed weights still possess the universal approximation property provided that approximated functions are univariate. But this phenomenon does not lay any restrictions on the number of neurons in the hidden layer. The more this number, the more the probability of the considered network to give precise results. In this note, we constructively prove that SLFNs with the fixed weight $1$ and two neurons in the hidden layer can approximate any continuous function on a compact subset of the real line. The applicability of this result is demonstrated in various numerical examples. Finally, we show that SLFNs with fixed weights cannot approximate all continuous multivariate functions.

17 pages, 5 figures, submitted; for associated SageMath worksheet, see https://sites.google.com/site/njguliyev/papers/monic-sigmoidal

References in corpus (3)

• Deep Learning in Neural Networks: An Overview
• A single hidden layer feedforward network with only one neuron in the hidden layer can approximate any univariate function
• Measure theoretic results for approximation by neural networks with limited weights

Cited by in corpus (13)

• Approximation capability of two hidden layer feedforward neural networks with fixed weights
• Two-hidden-layer Feedforward Neural Networks are Universal Approximators: A Constructive Approach
• Convergence in Orlicz spaces by means of the multivariate max-product neural network operators of the Kantorovich type and applications
• Full error analysis for the training of deep neural networks
• Efficient approximation of high-dimensional functions with neural networks
• Extreme learning machine collocation for the numerical solution of elliptic PDEs with sharp gradients
• On the approximation of functions by tanh neural networks
• Neutron star envelopes with machine learning: a single-hidden-layer neural network application
• Simmering: Sufficient is better than optimal for training neural networks
• Trainable and Explainable Simplicial Map Neural Networks
• Negative results for approximation using single layer and multilayer feedforward neural networks
• On shallow feedforward neural networks with inputs from a topological space
• Constructive approximation of continuous interval-valued functions