2 papers
cs.LG2026
Finite-Width Neural Tangent Kernels from Feynman Diagrams
Max Guillen, Philipp Misof, Jan E. Gerken
Neural tangent kernels (NTKs) are a powerful tool for analyzing deep, non-linear neural networks. In the infinite-width limit, NTKs can easily be computed for most common architect…
cs.LG2026
Criticality and Saturation in Orthogonal Neural Networks
Max Guillen, Jan E. Gerken
It has been known for a long time that initializing weight matrices to be orthogonal instead of having i.i.d. Gaussian components can improve training performance. This phenomenon…