Cartridges at Scale: Training Modular KV Caches over Large Document Collections

Momchil Hardalov, Gonzalo Iglesias, Adrià de Gispert

Large Language Models can reason over long contexts, yet prefilling millions of tokens is wasteful as much of the content remains static across queries. Cartridges address this by…