1 citations · 1 across the 6 of their papers we have counts for
Showing 2025Show all
3 papers · 1 filter
cs.DC2025
STAR: Decode-Phase Rescheduling for LLM Inference
Zhibin Wang, Zetao Hong, Xue Li +8
Large Language Model (LLM) inference has emerged as a fundamental paradigm, however, variations in output length cause severe workload imbalance in the decode phase, particularly f…
cs.LG2025
CoDec: Prefix-Shared Decoding Kernel for LLMs
Zhibin Wang, Rui Ning, Chao Fang +12
Prefix-sharing among multiple prompts presents opportunities to combine the operations of the shared prefix, while attention computation in the decode stage, which becomes a critic…
cs.DC2025
Echo: Efficient Co-Scheduling of Hybrid Online-Offline Tasks for Large Language Model Serving
Zhibin Wang, Shipeng Li, Xue Li +7
Large language models have been widely deployed in various applications, encompassing both interactive online tasks and batched offline tasks. Given the burstiness and latency sens…