Fully Deterministic Shallow Neural Networks with Localized Activations. Part I: Construction and Optimal Sobolev Approximation

Abstract

We construct shallow neural networks with fixed Gaussian and tanh-difference activations on bounded Lipschitz domains $\Omega \subset {\mathbb{R}}^d$, $d\ge 2$. For each $s>0$, explicit deterministic rules prescribe all hidden parameters independently of the target $u\in H^s(\Omega )$. At frequency resolution $M$, the $N_M\asymp M^d$ prescribed features span a linear space $V_M$. For every sufficiently large integer $N$, we choose $M\asymp N^{1/d}$ with $N_M\le N$ and construct a bounded linear operator $A_N:H^s(\Omega )\to V_M$ such that \[ \|u-A_Nu\|_{H^m(Ω)} \le C N^{-(s-m)/d}\|u\|_{H^s(Ω)} \] for every $u\in H^s(\Omega )$ and every integer $0\le m<s$, with $C$ independent of $u$, $N$, and $m$. Sobolev width lower bounds show that this rate is optimal among linear spaces of dimension at most $N$ chosen independently of the target. We use explicit spherical cubature nodes for the directions and Gauss-Legendre nodes or equispaced midpoints for the offsets. We show that the exterior offset margin can shrink to zero as $M\to \infty$ while preserving the approximation rate. A margin beyond ${\sup }_{x\in \Omega }|x|$ of order $M^{-1}\log M$ for the Gaussian and $M^{-1}(\log M{)}^2$ for the tanh difference suffices to truncate the ridge integral and remove exterior midpoint nodes. Numerical experiments in two and three dimensions examine $L^2$ and $H^1$ approximation.