GroupKV: Hierarchical KV Cache Management for Long-Context Diffusion LLM Inference

Jinhao Wang, Zhexin Hu, Kangjie Zhou +2

Diffusion large language models (dLLMs) are emerging as a promising generative paradigm that complements autoregressive decoding. In long-context settings, KV cache bloat and offlo…