6 citations · 9 across the 4 of their papers we have counts for
4 papers
DynaResize: Runtime GPU Reallocation for Disaggregated LLM Post-Training
Hanlin Du, Zhiyuan Yan, Yungang Bao +1
RL-based LLM post-training increasingly disaggregates Rollout and Training across separate GPU resources, but static GPU partitioning suffers from severe pipeline bubbles under lon…
DDiT: Dynamic Resource Allocation for Diffusion Transformer Model Serving
Heyang Huang, Cunchen Hu, Jiaqi Zhu +7
The Text-to-Video (T2V) model aims to generate dynamic and expressive videos from textual prompts. The generation pipeline typically involves multiple modules, such as language enc…
MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
Cunchen Hu, Heyang Huang, Junhao Hu +8
Large language model (LLM) serving has transformed from stateless to stateful systems, utilizing techniques like context caching and disaggregated inference. These optimizations ex…
Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads
Cunchen Hu, Heyang Huang, Liangliang Xu +9
Transformer-based large language model (LLM) inference serving is now the backbone of many cloud services. LLM inference consists of a prefill phase and a decode phase. However, ex…