38 citations · 73 across the 9 of their papers we have counts for
Showing 2019Show all
2 papers · 1 filter
cs.LG2019★ 3 cited
BackPACK: Packing more into backprop
Felix Dangel, Frederik Kunstner, Philipp Hennig
Automatic differentiation frameworks are optimized for exactly one thing: computing the average mini-batch gradient. Yet, other quantities such as the variance of the mini-batch gr…
cs.LG2019★ 38 cited
Limitations of the Empirical Fisher Approximation for Natural Gradient Descent
Frederik Kunstner, Lukas Balles, Philipp Hennig
Natural gradient descent, which preconditions a gradient descent update with the Fisher information matrix of the underlying statistical model, is a way to capture partial second-o…