Characterization of the Variation Spaces Corresponding to Shallow Neural Networks

Abstract

We study the variation space corresponding to a dictionary of functions in $L^2(\Omega )$ for a bounded domain $\Omega \subset {\mathbb{R}}^d$. Specifically, we compare the variation space, which is defined in terms of a convex hull with related notions based on integral representations. This allows us to show that three important notions relating to the approximation theory of shallow neural networks, the Barron space, the spectral Barron space, and the Radon BV space, are actually variation spaces with respect to certain natural dictionaries.

References in corpus (1)

• Approximation and learning by greedy algorithms

Cited by in corpus (4)

• Near-Minimax Optimal Estimation With Shallow ReLU Neural Networks
• Approximation Properties of Deep ReLU CNNs
• Sharp Lower Bounds on the Approximation Rate of Shallow Neural Networks
• Stationary Density Estimation of Itô Diffusions Using Deep Learning