3 papers
cs.CL2024
Automatic Pruning of Fine-tuning Datasets for Transformer-based Language Models
Mohammadreza Tayaranian, Seyyed Hasan Mozafari, Brett H. Meyer +2
Transformer-based language models have shown state-of-the-art performance on a variety of natural language understanding tasks. To achieve this performance, these models are first…
cs.CV2024
Faster Inference of Integer SWIN Transformer by Removing the GELU Activation
Mohammadreza Tayaranian, Seyyed Hasan Mozafari, James J. Clark +2
SWIN transformer is a prominent vision transformer model that has state-of-the-art accuracy in image classification tasks. Despite this success, its unique architecture causes slow…
cs.CL2022
Efficient Fine-Tuning of Compressed Language Models with Learners
Danilo Vucetic, Mohammadreza Tayaranian, Maryam Ziaeefard +3
Fine-tuning BERT-based models is resource-intensive in memory, computation, and time. While many prior works aim to improve inference efficiency via compression techniques, e.g., p…