Mathematical analysis of one-layer neural network with fixed biases, a new activation function and other observations

Fabricio Macià, Shu Nakamura

We analyze a simple one-hidden-layer neural network with ReLU activation functions and fixed biases, with one-dimensional input and output. We study both continuous and discrete ve…