Showing cs.CLShow all
3 papers · 1 filter
cs.CL2026
EpiCache: Episodic KV Cache Management for Long-Term Conversation on Resource-Constrained Environments
Minsoo Kim, Arnav Kundu, Han-Byul Kim +2
Modern large language models (LLMs) extend context lengths to millions of tokens, enabling coherent, personalized responses grounded in long conversational history. However, the Ke…
cs.CL2026
TIDE: Every Layer Knows the Token Beneath the Context
Ajay Jaiswal, Lauren Hannah, Han-Byul Kim +3
We revisit a universally accepted but under-examined design choice in every modern LLM: a token index is looked up once at the input embedding layer and then permanently discarded.…
cs.CL2025
Mirror Speculative Decoding: Breaking the Serial Barrier in LLM Inference
Nikhil Bhendawade, Kumari Nishu, Arnav Kundu +3
Speculative decoding accelerates LLM inference by using a draft model to look ahead, but gains are capped by the cost of autoregressive draft generation: increasing draft size elev…