2 papers
cs.DC2026
Fast and Memory Efficient Offload Training Framework with Hybrid XPU Computation
Zhiyi Yao, Zuning Liang, Yuedong Xu +3
With the ever-growing size of deep learning models, GPU memory is prone to being insufficient during training. A prominent approach is ZeRO-Offload, which moves the optimizer state…
cs.DC2026
DualDecoder: Accelerate Long Context LLM Inference by Predictive Prefetch
Zuning Liang, Zhiyi Yao, Qi Chen +6
Long-context inference is becoming a fundamental capability for modern LLM serving, especially driven by emerging agentic applications. Yet it faces a severe memory wall that the K…