Free Probability for predicting the performance of feed-forward fully connected neural networks

Reda Chhaibi, Tariq Daouda, Ezechiel Kahn

Gradient descent during the learning process of a neural network can be subject to many instabilities. The spectral density of the Jacobian is a key component for analyzing stabili…