Showing 2025 · cs.CLShow all
2 papers · 2 filters
cs.CL2025
Mirror Speculative Decoding: Breaking the Serial Barrier in LLM Inference
Nikhil Bhendawade, Kumari Nishu, Arnav Kundu +3
Speculative decoding accelerates LLM inference by using a draft model to look ahead, but gains are capped by the cost of autoregressive draft generation: increasing draft size elev…
cs.CL2025
EpiCache: Episodic KV Cache Management for Long-Term Conversation on Resource-Constrained Environments
Minsoo Kim, Arnav Kundu, Han-Byul Kim +2
Modern large language models (LLMs) extend context lengths to millions of tokens, enabling coherent, personalized responses grounded in long conversational history. However, the Ke…