PackInfer: Compute- and I/O-Efficient Attention for Batched LLM Inference

Rui Ning, Wei Zhang, Fan Lai

Attention efficiency is critical to large language model (LLM) inference. While prior advances optimize attention execution for individual requests (e.g., FlashAttention), producti…