3 papers
cs.LG2026
ScoutAttention: Efficient KV Cache Offloading via Layer-Ahead CPU Pre-computation for LLM Inference
Qiuyang Zhang, Kai Zhou, Ding Tang +5
Large language models encounter critical GPU memory capacity constraints during long-context inference, where KV cache memory consumption severely limits decode batch sizes. While…
cs.DC2025
H2:Towards Efficient Large-Scale LLM Training on Hyper-Heterogeneous Cluster over 1,000 Chips
Ding Tang, Jiecheng Zhou, Jiakai Hu +5
Recent advancements in large language models (LLMs) necessitate extensive computational resources, prompting the use of diverse hardware accelerators from multiple vendors. However…
cs.LG2025
Towards Efficient Pre-training: Exploring FP4 Precision in Large Language Models
Jiecheng Zhou, Ding Tang, Rong Fu +8
The burgeoning computational demands for training large language models (LLMs) necessitate efficient methods, including quantized training, which leverages low-bit arithmetic opera…