110 citations · 256 across the 18 of their papers we have counts for
4 papers · 1 filter
Finding Everything within Random Binary Networks
Kartik Sreenivasan, Shashank Rajput, Jy-yong Sohn +1
A recent work by Ramanujan et al. (2020) provides significant empirical evidence that sufficiently overparameterized, random neural networks contain untrained subnetworks that achi…
Minibatch vs Local SGD with Shuffling: Tight Convergence Bounds and Beyond
Chulhee Yun, Shashank Rajput, Suvrit Sra
In distributed learning, local SGD (also known as federated averaging) and its simple baseline minibatch SGD are widely studied optimization methods. Most existing analyses of thes…
An Exponential Improvement on the Memorization Capacity of Deep Threshold Networks
Shashank Rajput, Kartik Sreenivasan, Dimitris Papailiopoulos +1
It is well known that modern deep neural networks are powerful enough to memorize datasets even when the labels have been randomized. Recently, Vershynin (2020) settled a long stan…
Permutation-Based SGD: Is Random Optimal?
Shashank Rajput, Kangwook Lee, Dimitris Papailiopoulos
A recent line of ground-breaking results for permutation-based SGD has corroborated a widely observed phenomenon: random permutations offer faster convergence than with-replacement…