2 papers
cs.LG2026
HERALD: High-Throughput Block Diffusion LLM Serving via CPU-GPU Cooperative KV Cache Retrieval
Omin Kwon, Doyeon Kim, Jongseok Park +3
The KV cache dominates GPU memory in long-context LLM serving, crowding out batch capacity and leaving GPU compute idle. Offloading the cache to CPU DRAM restores capacity, but the…
cs.LG2026
MAGE: All-[MASK] Block Already Knows Where to Look in Block Diffusion LLM
Omin Kwon, Yeonjae Kim, Doyeon Kim +3
Block diffusion LLMs are an emerging paradigm for parallel language generation, but their KV caching makes memory access the dominant bottleneck in long-context inference. Sparse a…