42 citations · 83 across the 2 of their papers we have counts for
2 papers
cs.LG2018★ 42 cited
On the Computational Inefficiency of Large Batch Sizes for Stochastic Gradient Descent
Noah Golmant, Nikita Vemuri, Zhewei Yao +5
Increasing the mini-batch size for stochastic gradient descent offers significant opportunities to reduce wall-clock training time, but there are a variety of theoretical and syste…
cs.CV2017★ 41 cited
Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions
Bichen Wu, Alvin Wan, Xiangyu Yue +6
Neural networks rely on convolutions to aggregate spatial information. However, spatial convolutions are expensive in terms of model size and computation, both of which grow quadra…