111 citations · 111 across the 6 of their papers we have counts for
5 papers · 1 filter
KV-Kaizen: Learning Context-Adaptive Cache Compression Choices
Joao Monteiro, Louis Béthune, Anastasiia Filippova +3
As the context size of text processed with an LLM grows, the size of KV caches can outstrip the memory allocated for the original model weights. This impacts LLM throughput negativ…
KV-Lingo: Learning KV-Cache Translators with Distillation
Valérie Castin, Keitaro Sakamoto, Anastasiia Filippova +3
Large language models represent context with a key-value (KV) cache. Caches are model-specific: for the same text, models with different architectures or weights produce incompatib…
Scaling Categorical Flow Maps
Oscar Davis, Anastasiia Filippova, Pierre Ablin +4
Continuous diffusion and flow matching models could represent a powerful alternative to autoregressive approaches for language modelling (LM), as they unlock a host of advantages c…
Stochastic KV Routing: Enabling Adaptive Depth-Wise Cache Sharing
Anastasiia Filippova, David Grangier, Marco Cuturi +1
Serving transformer language models with high throughput requires caching Key-Values (KVs) to avoid redundant computation during autoregressive generation. The memory footprint of…
Optimal Splitting of Language Models from Mixtures to Specialized Domains
Skyler Seto, Pierre Ablin, Anastasiia Filippova +4
Language models achieve impressive performance on a variety of knowledge, language, and reasoning tasks due to the scale and diversity of pretraining data available. The standard t…