180 citations · 242 across the 4 of their papers we have counts for
7 papers
TIRAMISU: A Polyhedral Compiler for Dense and Sparse Deep Learning
Riyadh Baghdadi, Abdelkader Nadir Debbagh, Kamel Abdous +5
In this paper, we demonstrate a compiler that can optimize sparse and recurrent neural networks, both of which are currently outside of the scope of existing neural network compile…
What is the State of Neural Network Pruning?
Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle +1
Neural network pruning---the task of reducing the size of a network by removing parameters---has been the subject of a great deal of work in recent years. We provide a meta-analysi…
Comparing Rewinding and Fine-tuning in Neural Network Pruning
Alex Renda, Jonathan Frankle, Michael Carbin
Many neural network pruning algorithms proceed in three steps: train the network to completion, remove unwanted structure to compress the network, and retrain the remaining structu…
The Early Phase of Neural Network Training
Jonathan Frankle, David J. Schwab, Ari S. Morcos
Recent studies have shown that many important aspects of neural network learning take place within the very earliest iterations or epochs of training. For example, sparse, trainabl…
Linear Mode Connectivity and the Lottery Ticket Hypothesis
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel M. Roy +1
We study whether a neural network optimizes to the same, linearly connected minimum under different samples of SGD noise (e.g., random data order and augmentation). We find that st…
Dissecting Pruned Neural Networks
Jonathan Frankle, David Bau
Pruning is a standard technique for removing unnecessary structure from a neural network to reduce its storage footprint, computational demands, or energy consumption. Pruning can…