5 papers · 1 filter
KV-Lingo: Learning KV-Cache Translators with Distillation
Valérie Castin, Keitaro Sakamoto, Anastasiia Filippova +3
Large language models represent context with a key-value (KV) cache. Caches are model-specific: for the same text, models with different architectures or weights produce incompatib…
LaCy: What Small Language Models Can and Should Learn is Not Just a Question of Loss
Szilvia Ujváry, Louis Béthune, Pierre Ablin +3
Language models have consistently grown to compress more world knowledge into their parameters, but the knowledge that can be pretrained into them is upper-bounded by their paramet…
MoSE: Hierarchical Self-Distillation Enhances Early Layer Embeddings
Andrea Gurioli, Federico Pennino, João Monteiro +1
Deploying language models often requires navigating accuracy vs. performance trade-offs to meet latency constraints while preserving utility. Traditional model distillation reduces…
Explain-Query-Test: Self-Evaluating LLMs Via Explanation and Comprehension Discrepancy
Saeid Asgari Taghanaki, Joao Monteiro
Large language models (LLMs) have demonstrated remarkable proficiency in generating detailed and coherent explanations of complex concepts. However, the extent to which these model…
XC-Cache: Cross-Attending to Cached Context for Efficient LLM Inference
João Monteiro, Étienne Marcotte, Pierre-André Noël +5
In-context learning (ICL) approaches typically leverage prompting to condition decoder-only language model generation on reference information. Just-in-time processing of a context…