341 citations · 360 across the 10 of their papers we have counts for
10 papers
ELSA: Partial Weight Freezing for Overhead-Free Sparse Network Deployment
Paniz Halvachi, Alexandra Peste, Dan Alistarh +1
We present ELSA, a practical solution for creating deep networks that can easily be deployed at different levels of sparsity. The core idea is to embed one or more sparse networks…
Accurate Neural Network Pruning Requires Rethinking Sparse Optimization
Denis Kuznedelev, Eldar Kurtic, Eugenia Iofinova +3
Obtaining versions of deep neural networks that are both highly-accurate and highly-sparse is one of the main challenges in the area of model compression, and several high-performa…
Knowledge Distillation Performs Partial Variance Reduction
Mher Safaryan, Alexandra Peste, Dan Alistarh
Knowledge distillation is a popular approach for enhancing the performance of ''student'' models, with lower representational capacity, by taking advantage of more powerful ''teach…
Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures
Eugenia Iofinova, Alexandra Peste, Dan Alistarh
Pruning - that is, setting a significant subset of the parameters of a neural network to zero - is one of the most popular methods of model compression. Yet, several recent works h…
CrAM: A Compression-Aware Minimizer
Alexandra Peste, Adrian Vladu, Eldar Kurtic +2
Deep neural networks (DNNs) often have to be compressed, via pruning and/or quantization, before they can be deployed in practical settings. In this work we propose a new compressi…
How Well Do Sparse Imagenet Models Transfer?
Eugenia Iofinova, Alexandra Peste, Mark Kurtz +1
Transfer learning is a classic paradigm by which models pretrained on large "upstream" datasets are adapted to yield good results on "downstream" specialized datasets. Generally, m…
SSSE: Efficiently Erasing Samples from Trained Machine Learning Models
Alexandra Peste, Dan Alistarh, Christoph H. Lampert
The availability of large amounts of user-provided data has been key to the success of machine learning for many real-world tasks. Recently, an increasing awareness has emerged tha…
AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks
Alexandra Peste, Eugenia Iofinova, Adrian Vladu +1
The increasing computational requirements of deep neural networks (DNNs) have led to significant interest in obtaining DNN models that are sparse, yet accurate. Recent work has inv…
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
Torsten Hoefler, Dan Alistarh, Tal Ben-Nun +2
The growing energy and performance costs of deep learning have driven the community to reduce the size of neural networks by selectively pruning components. Similarly to their biol…
Learning in Variational Autoencoders with Kullback-Leibler and Renyi Integral Bounds
Septimia Sârbu, Riccardo Volpi, Alexandra Peşte +1
In this paper we propose two novel bounds for the log-likelihood based on Kullback-Leibler and the Rényi divergences, which can be used for variational inference and in particular…