2 citations · 3 across the 6 of their papers we have counts for
3 papers · 1 filter
Frobenius-Type Norms and Inner Products of Matrices and Linear Maps with Applications to Neural Network Training
Roland Herzog, Frederik Köhne, Leonie Kreis +1
The Frobenius norm is a frequent choice of norm for matrices. In particular, the underlying Frobenius inner product is typically used to evaluate the gradient of an objective with…
Adaptive Step Sizes for Preconditioned Stochastic Gradient Descent
Frederik Köhne, Leonie Kreis, Anton Schiela +1
This paper proposes a novel approach to adaptive step sizes in stochastic gradient descent (SGD) by utilizing quantities that we have identified as numerically traceable -- the Lip…
SensLI: Sensitivity-Based Layer Insertion for Neural Networks
Leonie Kreis, Evelyn Herberg, Frederik Köhne +2
The training of neural networks requires tedious and often manual tuning of the network architecture. We propose a systematic approach to inserting new layers during the training p…