Mitigating Outlier Activations in Low-Precision Fine-Tuning of Language Models

Alireza Ghaffari, Justin Yu, Mahsa Ghazvini Nejad +3

Low-precision fine-tuning of language models has gained prominence as a cost-effective and energy-efficient approach to deploying large-scale models in various applications. Howeve…