Two-hidden-layer Feedforward Neural Networks are Universal Approximators: A Constructive Approach

Abstract

It is well known that Artificial Neural Networks are universal approximators. The classical result proves that, given a continuous function on a compact set on an n-dimensional space, then there exists a one-hidden-layer feedforward network which approximates the function. Such result proves the existence, but it does not provide a method for finding it. In this paper, a constructive approach to the proof of this property is given for the case of two-hidden-layer feedforward networks. This approach is based on an approximation of continuous functions by simplicial maps. Once a triangulation of the space is given, a concrete architecture and set of weights can be obtained. The quality of the approximation depends on the refinement of the covering of the space by simplicial complexes.

References in corpus (8)

• On the Expressive Power of Deep Learning: A Tensor Analysis
• Approximating Continuous Functions by ReLU Nets of Minimal Width
• On the approximation by single hidden layer feedforward neural networks with fixed weights
• Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networks
• Benefits of depth in neural networks
• On the Expressive Efficiency of Sum Product Networks
• On the Expressive Power of Deep Polynomial Neural Networks
• Neural Networks Should Be Wide Enough to Learn Disconnected Decision Regions

Cited by in corpus (1)

• Trainable and Explainable Simplicial Map Neural Networks