Showing 2025Show all
3 papers · 1 filter
cs.LG2025
Compressed Decentralized Momentum Stochastic Gradient Methods for Nonconvex Optimization
Wei Liu, Anweshit Panda, Ujwal Pandey +6
In this paper, we design two compressed decentralized algorithms for solving nonconvex stochastic optimization under two different scenarios. Both algorithms adopt a momentum techn…
cs.LG2025
DiaBlo: Diagonal Blocks Are Sufficient For Finetuning
Selcuk Gurses, Aozhong Zhang, Yanxia Deng +5
Fine-tuning is a critical step for adapting large language models (LLMs) to domain-specific downstream tasks. To mitigate the substantial computational and memory costs of full-mod…
cs.LG2025
CLoQ: Enhancing Fine-Tuning of Quantized LLMs via Calibrated LoRA Initialization
Yanxia Deng, Aozhong Zhang, Selcuk Gurses +3
Fine-tuning large language models (LLMs) using low-rank adaptation (LoRA) has become a highly efficient approach for downstream tasks, particularly in scenarios with limited comput…