32 citations · 75 across the 10 of their papers we have counts for
Showing 2021 · cs.LGShow all
2 papers · 2 filters
cs.LG2021
Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping
James Martens, Andy Ballard, Guillaume Desjardins +4
Using an extended and formalized version of the Q/C map analysis of Poole et al. (2016), along with Neural Tangent Kernel theory, we identify the main pathologies present in deep n…
cs.LG2021★ 1 cited
On the validity of kernel approximations for orthogonally-initialized neural networks
James Martens
In this note we extend kernel function approximation results for neural networks with Gaussian-distributed weights to single-layer networks initialized using Haar-distributed rando…