1.3k citations · 2.1k across the 35 of their papers we have counts for
9 papers · 1 filter
The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models
Tianlong Chen, Jonathan Frankle, Shiyu Chang +4
The computer vision world has been re-gaining enthusiasm in various pre-trained models, including both classical ImageNet supervised pre-training and recently emerged self-supervis…
DiffTune: Optimizing CPU Simulator Parameters with Learned Differentiable Surrogates
Alex Renda, Yishen Chen, Charith Mendis +1
CPU simulators are useful tools for modeling CPU execution behavior. However, they suffer from inaccuracies due to the cost and complexity of setting their fine-grained parameters,…
Pruning Neural Networks at Initialization: Why are We Missing the Mark?
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel M. Roy +1
Recent work has explored the possibility of pruning neural networks at initialization. We assess proposals for doing so: SNIP (Lee et al., 2019), GraSP (Wang et al., 2020), SynFlow…
The Lottery Ticket Hypothesis for Pre-trained BERT Networks
Tianlong Chen, Jonathan Frankle, Shiyu Chang +4
In natural language processing (NLP), enormous pre-trained models like BERT have become the standard starting point for training on a range of downstream tasks, and similar trends…
Simplifying Dependent Reductions in the Polyhedral Model
Cambridge Yang, Eric Atkinson, Michael Carbin
A Reduction -- an accumulation over a set of values, using an associative and commutative operator -- is a common computation in many numerical computations, including scientific c…
: Computable Semantics for Differentiable Programming with Higher-Order Functions and Datatypes
Benjamin Sherman, Jesse Michel, Michael Carbin
Deep learning is moving towards increasingly sophisticated optimization objectives that employ higher-order functions, such as integration, continuous optimization, and root-findin…