2 citations · 2 across the 4 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.LG2023
Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC
Wu Lin, Felix Dangel, Runa Eschenhagen +4
Second-order methods such as KFAC can be useful for neural net training. However, they are often memory-inefficient since their preconditioning Kronecker factors are dense, and num…
cs.LG2023★ 2 cited
Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures
Runa Eschenhagen, Alexander Immer, Richard E. Turner +2
The core components of many modern neural network architectures, such as transformers, convolutional, or graph neural networks, can be expressed as linear layers with $\textit{weig…