3 papers
cs.DC2025
RServe: Overlapping Encoding and Prefill for Efficient LMM Inference
Tianyu Guo, Tianming Xu, Xianjie Chen +3
Large multimodal models (LMMs) typically employ an encoding module to transform multimodal data inputs into embeddings, which are then fed to language models for further processing…
cs.CL2025
EFIM: Efficient Serving of LLMs for Infilling Tasks with Improved KV Cache Reuse
Tianyu Guo, Hande Dong, Yichong Leng +4
Large language models (LLMs) are often used for infilling tasks, which involve predicting or generating missing information in a given text. These tasks typically require multiple…
cs.DC2025
gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling
Tianyu Guo, Xianwei Zhang, Jiangsu Du +3
Pipeline parallelism has emerged as a predominant approach for deploying large language models (LLMs) across distributed nodes, owing to its lower communication overhead compared t…