Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Echo: Decoupling Inference and Training for Large-Scale RL Alignment on Heterogeneous Swarms
Jie Xiao, Changyuan Fan, Qingnan Ren +6
Modern RL-based post-training for large language models (LLMs) co-locate trajectory sampling and policy optimisation on the same GPU cluster, forcing the system to switch between i…
cs.LG2024
SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget
Zihao Wang, Bin Cui, Shaoduo Gan
Optimizing the Key-Value (KV) cache of the Large Language Model (LLM) has been considered critical to saving the cost of inference. Most of the existing KV-cache compression algori…