Expressivity of Deep Neural Networks
arXiv:2007.04759
Abstract
In this review paper, we give a comprehensive overview of the large variety of approximation results for neural networks. Approximation rates for classical function spaces as well as benefits of deep neural networks over shallow ones for specifically structured function classes are discussed. While the mainbody of existing results is for general feedforward architectures, we also depict approximation results for convolutional, residual and recurrent neural networks.
arXiv commentThis review paper will appear as a book chapter in the book "Theory of Deep Learning" by Cambridge University Press
Cited by in corpus (6)
- The Modern Mathematics of Deep Learning
- Review: Deep Learning in Electron Microscopy
- Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problems
- Registration-based model reduction of parameterized two-dimensional conservation laws
- Approximation of Smoothness Classes by Deep Rectifier Networks
- Nonclosedness of Sets of Neural Networks in Sobolev Spaces