849 citations · 1.1k across the 20 of their papers we have counts for
5 papers · 1 filter
A Cookbook of Self-Supervised Learning
Randall Balestriero, Mark Ibrahim, Vlad Sobal +16
Self-supervised learning, dubbed the dark matter of intelligence, is a promising path to advance machine learning. Yet, much like cooking, training SSL methods is a delicate art wi…
Is Multi-Task Learning an Upper Bound for Continual Learning?
Zihao Wu, Huy Tran, Hamed Pirsiavash +1
Continual and multi-task learning are common machine learning approaches to learning from multiple tasks. The existing works in the literature often assume multi-task learning as a…
PRANC: Pseudo RAndom Networks for Compacting deep models
Parsa Nooralinejad, Ali Abbasi, Soroush Abbasi Koohpayegani +4
We demonstrate that a deep model can be reparametrized as a linear combination of several randomly initialized and frozen deep models in the weight space. During training, we seek…
Sparsity and Heterogeneous Dropout for Continual Learning in the Null Space of Neural Activations
Ali Abbasi, Parsa Nooralinejad, Vladimir Braverman +2
Continual/lifelong learning from a non-stationary input data stream is a cornerstone of intelligence. Despite their phenomenal performance in a wide variety of applications, deep n…
Amenable Sparse Network Investigator
Saeed Damadi, Erfan Nouri, Hamed Pirsiavash
We present "Amenable Sparse Network Investigator" (ASNI) algorithm that utilizes a novel pruning strategy based on a sigmoid function that induces sparsity level globally over the…