LoopKV: Loop-Aware KV Cache Compression for Serving Looped Transformers

Tianle Zhou, Yang Zhang, Tianyu Wu +3

Looped transformers apply a shared block several times per token to improve accuracy without adding parameters. For models such as Ouro that keep a separate key-value (KV) cache pe…