SparseInfer: Training-free Prediction of Activation Sparsity for Fast LLM Inference

Jiho Shin, Hoeseok Yang, Youngmin Yi

Leveraging sparsity is crucial for optimizing large language model inference. however, modern LLMs employing SiLU as their activation function exhibit minimal activation sparsity.…