1 citations · 1 across the 1 of their papers we have counts for
2 papers
eess.SY2025
Tail-Optimized Caching for LLM Inference
Wenxin Zhang, Yueying Li, Ciamac C. Moallemi +1
Prompt caching is critical for reducing latency and cost in LLM inference: OpenAI and Anthropic report up to 50-90% cost savings through prompt reuse. Despite its widespread succes…
cs.CL2025★ 1 cited
SplitReason: Learning To Offload Reasoning
Yash Akhauri, Anthony Fei, Chi-Chih Chang +3
Reasoning in large language models (LLMs) tends to produce substantially longer token generation sequences than simpler language modeling tasks. This extended generation length ref…