5 papers
PPL-Factory: Task-Aware and Budget-Aware Data Selection from Language Modeling to Reasoning
Hang Zhang, Warren J. Gross
Not all training samples contribute equally to large language model fine-tuning. Selecting informative training samples can reduce the computational cost while preserving downstrea…
Automatic Pruning of Fine-tuning Datasets for Transformer-based Language Models
Mohammadreza Tayaranian, Seyyed Hasan Mozafari, Brett H. Meyer +2
Transformer-based language models have shown state-of-the-art performance on a variety of natural language understanding tasks. To achieve this performance, these models are first…
Faster Inference of Integer SWIN Transformer by Removing the GELU Activation
Mohammadreza Tayaranian, Seyyed Hasan Mozafari, James J. Clark +2
SWIN transformer is a prominent vision transformer model that has state-of-the-art accuracy in image classification tasks. Despite this success, its unique architecture causes slow…
AdCorDA: Classifier Refinement via Adversarial Correction and Domain Adaptation
Lulan Shen, Ali Edalati, Brett Meyer +2
This paper describes a simple yet effective technique for refining a pretrained classifier network. The proposed AdCorDA method is based on modification of the training set and mak…
Robustness to distribution shifts of compressed networks for edge devices
Lulan Shen, Ali Edalati, Brett Meyer +2
It is necessary to develop efficient DNNs deployed on edge devices with limited computation resources. However, the compressed networks often execute new tasks in the target domain…