End-to-End Dynamic Sparsity for Resource-Adaptive LLM Inference

Yuhang Chen, Jinhao Duan, Ruichen Zhang +11

Large Language Models (LLMs) inference is typically deployed under a static resource assumption, where models execute a fixed computational graph regardless of the runtime environm…