Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Lethe: Layer- and Time-Adaptive KV Cache Pruning for Reasoning-Intensive LLM Serving
Hui Zeng, Daming Zhao, Pengfei Yang +5
Generative reasoning with large language models (LLMs) often involves long decoding sequences, leading to substantial memory and latency overheads from accumulating key-value (KV)…
cs.LG2025
Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decoding
StepFun, :, Bin Wang +195
Large language models (LLMs) face low hardware efficiency during decoding, especially for long-context reasoning tasks. This paper introduces Step-3, a 321B-parameter VLM with hard…