223 citations · 1.8k across the 31 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2023★ 3 cited
Grokking modular arithmetic
Andrey Gromov
We present a simple neural network that can learn modular arithmetic tasks and exhibits a sudden jump in generalization known as ``grokking''. Concretely, we present (i) fully-conn…
cs.LG2021★ 2 cited
Critical Initialization of Wide and Deep Neural Networks through Partial Jacobians: General Theory and Applications
Darshil Doshi, Tianyu He, Andrey Gromov
Deep neural networks are notorious for defying theoretical treatment. However, when the number of parameters in each layer tends to infinity, the network function is a Gaussian pro…