1 citations · 1 across the 8 of their papers we have counts for
4 papers · 1 filter
HERALD: High-Throughput Block Diffusion LLM Serving via CPU-GPU Cooperative KV Cache Retrieval
Omin Kwon, Doyeon Kim, Jongseok Park +3
The KV cache dominates GPU memory in long-context LLM serving, crowding out batch capacity and leaving GPU compute idle. Offloading the cache to CPU DRAM restores capacity, but the…
Uncovering Intra-expert Activation Sparsity for Efficient Mixture-of-Expert Model Execution
Jongseok Park, Sunga Kim, Zhenyu Gu +2
Mixture of Experts (MoE) architecture has become the standard for state-of-the-art large language models, owing to its computational efficiency through sparse expert activation. Ho…
Qrita: High-performance Top-k and Top-p using Pivot-based Truncation and Selection
Jongseok Park, Sunga Kim, Alvin Cheung +1
Despite their importance in model sampling, efficient implementation of Top-k and Top-p algorithms for large vocabularies remains a significant challenge. Existing approaches often…
Speculative Decoding: Performance or Illusion?
Xiaoxuan Liu, Jiaxiang Yu, Jongseok Park +2
Speculative decoding (SD) has become a popular technique to accelerate Large Language Model (LLM) inference, yet its real-world effectiveness remains unclear as prior evaluations r…