WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning

Haozhe Hu, Hao Wu, Peiran Yin +3

Pruning is a promising approach for improving the efficiency of LLMs. Existing static structured pruning methods are hardware-friendly and can deliver practical throughput gains, b…