High Probability Derivative Bounds for Random tanh Neural Networks on a Hypercube

Josef Dick, Michael Feischl, Fabian Zehetgruber

We establish high-probability bounds for mixed input derivatives of wide random neural networks whose activation derivatives satisfy a factorial growth bound. Our main result speci…