1.4k citations · 1.5k across the 4 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2019★ 15 cited
How do infinite width bounded norm networks look in function space?
Pedro Savarese, Itay Evron, Daniel Soudry +1
We consider the question of what functions can be captured by ReLU networks with an unbounded number of units (infinite width), but where the overall network Euclidean norm (sum of…
cs.LG2019★ 50 cited
Augment your batch: better training with larger batches
Elad Hoffer, Tal Ben-Nun, Itay Hubara +3
Large-batch SGD is important for scaling training of deep neural networks. However, without fine-tuning hyperparameter schedules, the generalization of the model may be hampered. W…