11 citations · 43 across the 16 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.LG2023
Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency
Vithursan Thangarasa, Shreyas Saxena, Abhay Gupta +1
Recent research has focused on weight sparsity in deep neural network training to reduce FLOPs, aiming for improved efficiency (test accuracy w.r.t training FLOPs). However, sparse…
cs.LG2023★ 7 cited
SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models
Vithursan Thangarasa, Abhay Gupta, William Marshall +5
The pre-training and fine-tuning paradigm has contributed to a number of breakthroughs in Natural Language Processing (NLP). Instead of directly training on a downstream task, lang…