2 papers
cs.LG2025
Block-Diagonal LoRA for Eliminating Communication Overhead in Tensor Parallel LoRA Serving
Xinyu Wang, Jonas M. Kübler, Kailash Budhathoki +2
When serving a single base LLM with several different LoRA adapters simultaneously, the adapters cannot simply be merged with the base model's weights as the adapter swapping would…
cs.LG2025
A Proximal Operator for Inducing 2:4-Sparsity
Jonas M Kübler, Yu-Xiang Wang, Shoham Sabach +5
Recent hardware advancements in AI Accelerators and GPUs allow to efficiently compute sparse matrix multiplications, especially when 2 out of 4 consecutive weights are set to zero.…