High-Layer Attention Pruning with Rescaling

Songtao Liu, Peng Liu

Pruning is a highly effective approach for compressing large language models (LLMs), significantly reducing inference latency. However, conventional training-free structured prunin…