3 papers
cs.DC2026
PackInfer: Compute- and I/O-Efficient Attention for Batched LLM Inference
Rui Ning, Wei Zhang, Fan Lai
Attention efficiency is critical to large language model (LLM) inference. While prior advances optimize attention execution for individual requests (e.g., FlashAttention), producti…
cs.DC2025
JITServe: SLO-aware LLM Serving with Imprecise Request Information
Wei Zhang, Zhiyu Wu, Yi Mu +5
The integration of Large Language Models (LLMs) into applications ranging from interactive chatbots to multi-agent systems has introduced a wide spectrum of service-level objective…
cs.IR2024
Fine-Grained Embedding Dimension Optimization During Training for Recommender Systems
Qinyi Luo, Penghan Wang, Wei Zhang +8
Huge embedding tables in modern deep learning recommender models (DLRM) require prohibitively large memory during training and inference. This paper proposes FIITED, a system to au…