Towards Extreme Pruning of LLMs with Plug-and-Play Mixed Sparsity

Chi Xu, Gefei Zhang, Yantong Zhu +4

N:M structured pruning is essential for large language models (LLMs) because it can remove less important network weights and reduce the memory and computation requirements. Existi…