Asymmetric Virtual Memory Paging for Hybrid Mamba-Transformer Inference

An Xuan Nguyen

Hybrid language models like Jamba mix attention layers with State Space Models (SSMs), creating two memory cache types with opposite profiles: Key-Value (KV) caches grow linearly w…