4 papers · 1 filter
Sparse Gradient Compression for Fine-Tuning Large Language Models
David H. Yang, Mohammad Mohammadi Amiri, Tejaswini Pedapati +2
Fine-tuning large language models (LLMs) for downstream tasks has become increasingly crucial due to their widespread use and the growing availability of open-source models. Howeve…
Differentiable Prompt Learning for Vision Language Models
Zhenhan Huang, Tejaswini Pedapati, Pin-Yu Chen +1
Prompt learning is an effective way to exploit the potential of large-scale pre-trained foundational models. Continuous prompts parameterize context tokens in prompts by turning th…
Graph is all you need? Lightweight data-agnostic neural architecture search without training
Zhenhan Huang, Tejaswini Pedapati, Pin-Yu Chen +2
Neural architecture search (NAS) enables the automatic design of neural network models. However, training the candidates generated by the search algorithm for performance evaluatio…
From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers
Bharat Runwal, Tejaswini Pedapati, Pin-Yu Chen
Pretrained Language Models (PLMs) have become the de facto starting point for fine-tuning on downstream tasks. However, as model sizes continue to increase, traditional fine-tuning…