FLRC: Fine-grained Low-Rank Compressor for Efficient LLM Inference

Yu-Chen Lu, Chong-Yan Chen, Chi-Chih Chang +2

Although large language models (LLM) have achieved remarkable performance, their enormous parameter counts hinder deployment on resource-constrained hardware. Low-rank compression…