305 citations · 611 across the 20 of their papers we have counts for
3 papers · 1 filter
Weight-space symmetry in deep networks gives rise to permutation saddles, connected by equal-loss valleys across the loss landscape
Johanni Brea, Berfin Simsek, Bernd Illing +1
The permutation symmetry of neurons in each layer of a deep neural network gives rise not only to multiple equivalent global minima of the loss function, but also to first-order sa…
Learning in Volatile Environments with the Bayes Factor Surprise
Vasiliki Liakoni, Alireza Modirshanechi, Wulfram Gerstner +1
Surprise-based learning allows agents to rapidly adapt to non-stationary stochastic environments characterized by sudden changes. We show that exact Bayesian inference in a hierarc…
Biologically plausible deep learning -- but how far can we go with shallow networks?
Bernd Illing, Wulfram Gerstner, Johanni Brea
Training deep neural networks with the error backpropagation algorithm is considered implausible from a biological perspective. Numerous recent publications suggest elaborate model…