Unlocking Structured Pruning for Reasoning Language Models via Attention-Preserving MLP Trimming

Woosang Lim, Haneul Yoo, Vincent-Daniel Yun +1

Structured pruning reduces LLM inference costs by removing model components to produce smaller dense models. However, existing methods can cause reasoning performance to collapse o…