Sparse Gradient Compression for Fine-Tuning Large Language Models

David H. Yang, Mohammad Mohammadi Amiri, Tejaswini Pedapati +2

Fine-tuning large language models (LLMs) for downstream tasks has become increasingly crucial due to their widespread use and the growing availability of open-source models. Howeve…