145 citations · 600 across the 26 of their papers we have counts for
3 papers · 1 filter
Learning Fine-grained Parameter Sharing via Sparse Tensor Decomposition
Cem Üyük, Mike Lasby, Mohamed Yassin +2
Large neural networks achieve state-of-the-art performance on many tasks, yet their sheer size hinders deployment on resource-constrained devices. Among existing compression approa…
Towards Optimal Adapter Placement for Efficient Transfer Learning
Aleksandra I. Nowak, Otniel-Bogdan Mercea, Anurag Arnab +3
Parameter-efficient transfer learning (PETL) aims to adapt pre-trained models to new downstream tasks while minimizing the number of fine-tuned parameters. Adapters, a popular appr…
Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers
Abhimanyu Rajeshkumar Bambhaniya, Amir Yazdanbakhsh, Suvinay Subramanian +4
N:M Structured sparsity has garnered significant interest as a result of relatively modest overhead and improved efficiency. Additionally, this form of sparsity holds considerable…