1.3k citations · 2.2k across the 33 of their papers we have counts for
3 papers · 1 filter
Trade-offs of Local SGD at Scale: An Empirical Study
Jose Javier Gonzalez Ortiz, Jonathan Frankle, Mike Rabbat +2
As datasets and models become increasingly large, distributed training has become a necessary component to allow deep neural networks to train in reasonable amounts of time. Howeve…
What can linear interpolation of neural network loss landscapes tell us?
Tiffany Vlaar, Jonathan Frankle
Studying neural network loss landscapes provides insights into the nature of the underlying optimization problems. Unfortunately, loss landscapes are notoriously difficult to visua…
Studying the Consistency and Composability of Lottery Ticket Pruning Masks
Rajiv Movva, Jonathan Frankle, Michael Carbin
Magnitude pruning is a common, effective technique to identify sparse subnetworks at little cost to accuracy. In this work, we ask whether a particular architecture's accuracy-spar…