4 papers · 1 filter
Triage: Skip, Skim, or Refine Attention Tiles in Video Diffusion Transformers
Yeonjae Kim, Omin Kwon, Seong Hoon Seo +1
Attention over tens of thousands of visual tokens dominates the inference of video diffusion transformers (DiTs). Existing methods reduce this cost through sparsity, low precision,…
SketchSSM: Write to the Full State, Read from a Compact Sketch
Omin Kwon, JoongWon Shin, Minseo Kim +3
Hybrid-attention models replace most softmax attention layers with linear attention, reducing KV-cache growth and enabling larger decode batches where recurrent-state access become…
HERALD: High-Throughput Block Diffusion LLM Serving via CPU-GPU Cooperative KV Cache Retrieval
Omin Kwon, Doyeon Kim, Jongseok Park +3
The KV cache dominates GPU memory in long-context LLM serving, crowding out batch capacity and leaving GPU compute idle. Offloading the cache to CPU DRAM restores capacity, but the…
MAGE: All-[MASK] Block Already Knows Where to Look in Block Diffusion LLM
Omin Kwon, Yeonjae Kim, Doyeon Kim +3
Block diffusion LLMs are an emerging paradigm for parallel language generation, but their KV caching makes memory access the dominant bottleneck in long-context inference. Sparse a…