Showing cs.ARShow all
2 papers · 1 filter
cs.AR2026
Trillion-Parameter MoE in a Box: Decoupling Memory Provisioning with High-Bandwidth Flash
Pengfei Xia, Tuo Hao, Shengwei Li +5
An MoE appliance for trillion-parameter models at low concurrency must host terabytes of weights on one node and serve prefill and decode with fixed resources. Combining operator a…
cs.AR2026
Enabling High-Bandwidth Flash for Generative Recommendation Serving with Write-Aware KV Cache Policy
Danni Peng, Kai Wu, Tianyu Zuo +2
Generative recommendation (GR) systems increasingly leverage user-level KV cache reuse to avoid recomputing long user histories. However, the growing KV cache capacity and bandwidt…