Embedding Inequalities for Barron-type Spaces

Abstract

An important problem in machine learning theory is to understand the approximation and generalization properties of two-layer neural networks in high dimensions. To this end, researchers have introduced the Barron space ${\mathcal{B}}_s(\Omega )$ and the spectral Barron space ${\mathcal{F}}_s(\Omega )$, where the index $s\in [0,\infty )$ indicates the smoothness of functions within these spaces and $\Omega \subset {\mathbb{R}}^d$ denotes the input domain. However, the precise relationship between the two types of Barron spaces remains unclear. In this paper, we establish a continuous embedding between them as implied by the following inequality: for any $\delta \in (0,1),s\in {\mathbb{N}}^{+}$ and $f:\Omega \mapsto \mathbb{R}$, it holds that \[ δ\|f\|_{\mathcal{F}_{s-δ}(Ω)}\lesssim_s \|f\|_{\mathcal{B}_s(Ω)}\lesssim_s \|f\|_{\mathcal{F}_{s+1}(Ω)}. \] Importantly, the constants do not depend on the input dimension $d$, suggesting that the embedding is effective in high dimensions. Moreover, we also show that the lower and upper bound are both tight.

11 pages

References in corpus (8)

• The Finite Neuron Method and Convergence Analysis
• Norm-Based Capacity Control in Neural Networks
• Sharp Bounds on the Approximation Rates, Metric Entropy, and $n$-widths of Shallow Neural Networks
• Banach Space Representer Theorems for Neural Networks and Ridge Splines
• Some observations on high-dimensional partial differential equations with Barron data
• Complexity Measures for Neural Networks with General Activation Functions Using Path-based Norms
• On the Representation of Solutions to Elliptic PDEs in Barron Spaces
• Neural network approximation and estimation of classifiers with classification boundary in a Barron class