5 papers · 1 filter
Carryover Drafting: Recycling Rejected States for Speculative Decoding
Jahyun Koo, Sunghyeon Woo, Jaeeun Kil +4
Speculative decoding accelerates LLM inference by verifying multiple drafted tokens in parallel, allowing a single target forward pass to accept several tokens. By construction, ve…
ICaRus: Identical Cache Reuse for Efficient Multi Model Inference
Sunghyeon Woo, Jaeeun Kil, Hoseung Kim +9
Multi model inference has recently emerged as a prominent paradigm, particularly in the development of agentic AI systems. However, in such scenarios, each model must maintain its…
PrefillShare: A Shared Prefill Module for KV Reuse in Multi-LLM Disaggregated Serving
Sunghyeon Woo, Hoseung Kim, Sunghwan Shim +9
Multi-agent systems increasingly orchestrate multiple specialized language models to solve complex real-world problems, often invoking them over a shared context. This execution pa…
PaCA: Partial Connection Adaptation for Efficient Fine-Tuning
Sunghyeon Woo, Sol Namkung, Sunwoo Lee +3
Prior parameter-efficient fine-tuning (PEFT) algorithms reduce memory usage and computational costs of fine-tuning large neural network models by training only a few additional ada…
DropBP: Accelerating Fine-Tuning of Large Language Models by Dropping Backward Propagation
Sunghyeon Woo, Baeseong Park, Byeongwook Kim +4
Large language models (LLMs) have achieved significant success across various domains. However, training these LLMs typically involves substantial memory and computational costs du…