High-Order Approximation Rates for Shallow Neural Networks with Cosine and ReLU${}^k$ Activation Functions

Jonathan W. Siegel, Jinchao Xu

We study the approximation properties of shallow neural networks with an activation function which is a power of the rectified linear unit. Specifically, we consider the dependence…