Runtime Observability for Heterogeneous Attention Memory

Fanzhe Wei, Li Liu, Ziyang Wang +1

Modern models no longer keep a plain KV cache: latent caches, learned sparse selectors and recurrent states each carry the model's memory in a different form, and each fails differ…