849 citations · 1.1k across the 20 of their papers we have counts for
9 papers · 1 filter
Is Multi-Task Learning an Upper Bound for Continual Learning?
Zihao Wu, Huy Tran, Hamed Pirsiavash +1
Continual and multi-task learning are common machine learning approaches to learning from multiple tasks. The existing works in the literature often assume multi-task learning as a…
Backdoor Attacks on Vision Transformers
Akshayvarun Subramanya, Aniruddha Saha, Soroush Abbasi Koohpayegani +2
Vision Transformers (ViT) have recently demonstrated exemplary performance on a variety of vision tasks and are being used as an alternative to CNNs. Their design is based on a sel…
SimA: Simple Softmax-free Attention for Vision Transformers
Soroush Abbasi Koohpayegani, Hamed Pirsiavash
Recently, vision transformers have become very popular. However, deploying them in many applications is computationally expensive partly due to the Softmax layer in the attention b…
PRANC: Pseudo RAndom Networks for Compacting deep models
Parsa Nooralinejad, Ali Abbasi, Soroush Abbasi Koohpayegani +4
We demonstrate that a deep model can be reparametrized as a linear combination of several randomly initialized and frozen deep models in the weight space. During training, we seek…
A Simple Approach to Adversarial Robustness in Few-shot Image Classification
Akshayvarun Subramanya, Hamed Pirsiavash
Few-shot image classification, where the goal is to generalize to tasks with limited labeled data, has seen great progress over the years. However, the classifiers are vulnerable t…
Sparsity and Heterogeneous Dropout for Continual Learning in the Null Space of Neural Activations
Ali Abbasi, Parsa Nooralinejad, Vladimir Braverman +2
Continual/lifelong learning from a non-stationary input data stream is a cornerstone of intelligence. Despite their phenomenal performance in a wide variety of applications, deep n…
Amenable Sparse Network Investigator
Saeed Damadi, Erfan Nouri, Hamed Pirsiavash
We present "Amenable Sparse Network Investigator" (ASNI) algorithm that utilizes a novel pruning strategy based on a sigmoid function that induces sparsity level globally over the…
SimReg: Regression as a Simple Yet Effective Tool for Self-supervised Knowledge Distillation
K L Navaneet, Soroush Abbasi Koohpayegani, Ajinkya Tejankar +1
Feature regression is a simple way to distill large neural network models to smaller ones. We show that with simple changes to the network architecture, regression can outperform m…
A Fistful of Words: Learning Transferable Visual Models from Bag-of-Words Supervision
Ajinkya Tejankar, Maziar Sanjabi, Bichen Wu +4
Using natural language as a supervision for training visual recognition models holds great promise. Recent works have shown that if such supervision is used in the form of alignmen…