2 papers
cs.LG2026
Carryover Drafting: Recycling Rejected States for Speculative Decoding
Jahyun Koo, Sunghyeon Woo, Jaeeun Kil +4
Speculative decoding accelerates LLM inference by verifying multiple drafted tokens in parallel, allowing a single target forward pass to accept several tokens. By construction, ve…
cs.LG2026
ICaRus: Identical Cache Reuse for Efficient Multi Model Inference
Sunghyeon Woo, Jaeeun Kil, Hoseung Kim +9
Multi model inference has recently emerged as a prominent paradigm, particularly in the development of agentic AI systems. However, in such scenarios, each model must maintain its…