Global universality of the two-layer neural network with the $k$-rectified linear unit

Abstract

This paper concerns the universality of the two-layer neural network with the $k$-rectified linear unit activation function with $k=1,2,\dots$ with a suitable norm without any restriction on the shape of the domain. This type of result is called global universality, which extends the previous result for $k=1$ by the present authors. This paper covers $k$-sigmoidal functions as an application of the fundamental result on $k$-rectified linear unit functions.