MTServe: Efficient Serving for Generative Recommendation Models with Hierarchical Caches

Xin Wang, Chi Ma, Shaobin Chen +14

Generative recommendation (GR) offers superior modeling capabilities but suffers from prohibitive inference costs due to the repeated encoding of long user histories. While cross-r…