Dynamic HBM Repartitioning for Multi-Turn MoE Serving

Jinpyo Kim, Mingi Kwon, Younghoon Min +2

Long-running multi-turn requests accumulate reusable key-value (KV) state. Once this state exceeds a fixed GPU KV-cache allocation, serving systems evict reusable prefixes, repeat…