Pangu Light: Weight Re-Initialization for Pruning and Accelerating LLMs

Hanting Chen, Jiarui Qin, Jialong Guo +15

Large Language Models (LLMs) deliver state-of-the-art capabilities across numerous tasks, but their immense size and inference costs pose significant computational challenges for p…