54 citations · 161 across the 5 of their papers we have counts for
Showing cs.NEShow all
3 papers · 1 filter
cs.NE2019★ 54 cited
A Mean Field Theory of Batch Normalization
Greg Yang, Jeffrey Pennington, Vinay Rao +2
We develop a mean field theory for batch normalization in fully-connected feedforward neural networks. In so doing, we provide a precise characterization of signal propagation and…
cs.NE2019
Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation
Greg Yang
Several recent trends in machine learning theory and practice, from the design of state-of-the-art Gaussian Process to the convergence analysis of deep neural nets (DNNs) under sto…
cs.NE2017★ 32 cited
Mean Field Residual Networks: On the Edge of Chaos
Greg Yang, Samuel S. Schoenholz
We study randomly initialized residual networks using mean field theory and the theory of difference equations. Classical feedforward neural networks, such as those with tanh activ…