Approximation Theory for Neural Networks: Old and New

Soumendu Sundar Mukherjee, Himasish Talukdar

Universal approximation theorems provide a mathematical explanation for the expressive power of neural networks. They assert that, under mild conditions on the activation function,…