25 citations · 43 across the 5 of their papers we have counts for
5 papers
Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency
Vithursan Thangarasa, Shreyas Saxena, Abhay Gupta +1
Recent research has focused on weight sparsity in deep neural network training to reduce FLOPs, aiming for improved efficiency (test accuracy w.r.t training FLOPs). However, sparse…
SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models
Vithursan Thangarasa, Abhay Gupta, William Marshall +5
The pre-training and fine-tuning paradigm has contributed to a number of breakthroughs in Natural Language Processing (NLP). Instead of directly training on a downstream task, lang…
RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network
Vitaliy Chiley, Vithursan Thangarasa, Abhay Gupta +3
This work introduces RevSilo, the first reversible bidirectional multi-scale feature fusion module. Like other reversible methods, RevSilo eliminates the need to store hidden activ…
Approximating Wisdom of Crowds using K-RBMs
Abhay Gupta
An important way to make large training sets is to gather noisy labels from crowds of non experts. We propose a method to aggregate noisy labels collected from a crowd of workers o…
DAiSEE: Towards User Engagement Recognition in the Wild
Abhay Gupta, Arjun D'Cunha, Kamal Awasthi +1
We introduce DAiSEE, the first multi-label video classification dataset comprising of 9068 video snippets captured from 112 users for recognizing the user affective states of bored…