UniRank: Unified Rank Allocation for Low-Rank LLM Compression

Chao Han, Haozhe Hu, Yongjie Du +5

Low-rank decomposition is a promising compression paradigm for large language models (LLMs), yet its effectiveness hinges on rank budget allocation across weight matrices: uniform…