Enabling High-Bandwidth Flash for Generative Recommendation Serving with Write-Aware KV Cache Policy

Danni Peng, Kai Wu, Tianyu Zuo +2

Generative recommendation (GR) systems increasingly leverage user-level KV cache reuse to avoid recomputing long user histories. However, the growing KV cache capacity and bandwidt…