74 citations · 100 across the 8 of their papers we have counts for
9 papers
DataComp: In search of the next generation of multimodal datasets
Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang +31
Multimodal datasets are a critical component in recent breakthroughs such as Stable Diffusion and GPT-4, yet their design does not receive the same research attention as model arch…
The Role of Pre-training Data in Transfer Learning
Rahim Entezari, Mitchell Wortsman, Olga Saukh +3
The transfer learning paradigm of model pre-training and subsequent fine-tuning produces high-accuracy models. While most studies recommend scaling the pre-training size to benefit…
REPAIR: REnormalizing Permuted Activations for Interpolation Repair
Keller Jordan, Hanie Sedghi, Olga Saukh +2
In this paper we look into the conjecture of Entezari et al. (2021) which states that if the permutation invariance of neural networks is taken into account, then there is likely n…
Studying the impact of magnitude pruning on contrastive learning methods
Francesco Corti, Rahim Entezari, Sara Hooker +2
We study the impact of different pruning techniques on the representation learned by deep neural networks trained with contrastive loss functions. Our work finds that at high spars…
Understanding the effect of sparsity on neural networks robustness
Lukas Timpl, Rahim Entezari, Hanie Sedghi +2
This paper examines the impact of static sparsity on the robustness of a trained network to weight perturbations, data corruption, and adversarial examples. We show that, up to a c…
Deep Neural Network Pruning for Nuclei Instance Segmentation in Hematoxylin & Eosin-Stained Histological Images
Amirreza Mahbod, Rahim Entezari, Isabella Ellinger +1
Recently, pruning deep neural networks (DNNs) has received a lot of attention for improving accuracy and generalization power, reducing network size, and increasing inference speed…
The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks
Rahim Entezari, Hanie Sedghi, Olga Saukh +1
In this paper, we conjecture that if the permutation invariance of neural networks is taken into account, SGD solutions will likely have no barrier in the linear interpolation betw…
Class-dependent Compression of Deep Neural Networks
Rahim Entezari, Olga Saukh
Today's deep neural networks require substantial computation resources for their training, storage, and inference, which limits their effective use on resource-constrained devices.…
AVID: Adversarial Visual Irregularity Detection
Mohammad Sabokrou, Masoud Pourreza, Mohsen Fayyaz +4
Real-time detection of irregularities in visual data is very invaluable and useful in many prospective applications including surveillance, patient monitoring systems, etc. With th…