Showing cs.CLShow all
2 papers · 1 filter
cs.CL2024
Refining Salience-Aware Sparse Fine-Tuning Strategies for Language Models
Xinxin Liu, Aaron Thomas, Cheng Zhang +3
Parameter-Efficient Fine-Tuning (PEFT) has gained prominence through low-rank adaptation methods like LoRA. In this paper, we focus on sparsity-based PEFT (SPEFT), which introduces…
cs.CL2024
Scaling Laws For Mixed Quantization
Zeyu Cao, Boyang Gu, Cheng Zhang +5
Post-training quantization of Large Language Models (LLMs) has proven effective in reducing the memory and computational requirements for inference. In this study, we focus on a st…