Wide Neural Networks as a Baseline for the Computational No-Coincidence Conjecture

John Dunbar, Scott Aaronson

We establish that randomly initialized neural networks, with large width and a natural choice of hyperparameters, have nearly independent outputs exactly when their activation func…