4 papers
\k{appa}-LoRA: Condition Numbers Reveal Which LoRA Matrices Worth Updating
Jianghui Wang, Silong Yong, Francesco Orabona +3
Low-Rank Adaptation (LoRA) has become a widely adopted technique for efficient neural network fine-tuning, decomposing model updates into low-rank matrices. However, LoRA remains c…
Why Smaller Is Slower? Dimensional Misalignment in Compressed LLMs
Jihao Xin, Tian Lyu, Qilong Pan +2
Post-training compression reduces LLM parameter counts but often produces irregular tensor dimensions that degrade GPU performance -- a phenomenon we call \emph{dimensional misalig…
ACING: Actor-Critic for Instruction Learning in Black-Box LLMs
Salma Kharrat, Fares Fourati, Marco Canini
The effectiveness of Large Language Models (LLMs) in solving tasks depends significantly on the quality of their instructions, which often require substantial human effort to craft…
Where is the Testbed for my Federated Learning Research?
Janez BožiÄ, Amândio R. Faustino, Boris RadoviÄ +2
Progressing beyond centralized AI is of paramount importance, yet, distributed AI solutions, in particular various federated learning (FL) algorithms, are often not comprehensively…