131 citations · 165 across the 17 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2025
ReSpec: Towards Optimizing Speculative Decoding in Reinforcement Learning Systems
Qiaoling Chen, Zijun Liu, Peng Sun +6
Adapting large language models (LLMs) via reinforcement learning (RL) is often bottlenecked by the generation stage, which can consume over 75\% of the training time. Speculative d…
cs.LG2025
Semantic-Aware Scheduling for GPU Clusters with Large Language Models
Zerui Wang, Qinghao Hu, Ana Klimovic +4
Deep learning (DL) schedulers are pivotal in optimizing resource allocation in GPU clusters, but operate with a critical limitation: they are largely blind to the semantic context…
cs.LG2025
Rethinking Key-Value Cache Compression Techniques for Large Language Model Serving
Wei Gao, Xinyu Zhou, Peng Sun +2
Key-Value cache (\texttt{KV} \texttt{cache}) compression has emerged as a promising technique to optimize Large Language Model (LLM) serving. It primarily decreases the memory cons…