3 citations · 3 across the 5 of their papers we have counts for
5 papers
LLM Router: Rethinking Routing with Prefill Activations
Tanay Varshney, Annie Surla, Michelle Xu +5
Existing routers rely on semantic query features or handcrafted features, which often fail to capture model-specific failures or intrinsic task difficulty. We instead route using i…
KVzap: Fast, Adaptive, and Faithful KV Cache Pruning
Simon Jegou, Maximilian Jeblick
Growing context lengths in transformer-based language models have made the key-value (KV) cache a critical inference bottleneck. While many KV cache pruning methods have been propo…
Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution
Alessio Devoto, Maximilian Jeblick, Simon Jégou
Memory consumption of the Key-Value (KV) cache represents a major bottleneck for efficient large language model inference. While attention-score-based KV cache pruning shows promis…
H2O-Danube-1.8B Technical Report
Philipp Singer, Pascal Pfeiffer, Yauhen Babakhin +4
We present H2O-Danube, a series of small 1.8B language models consisting of H2O-Danube-1.8B, trained on 1T tokens, and the incremental improved H2O-Danube2-1.8B trained on an addit…
H2O Open Ecosystem for State-of-the-art Large Language Models
Arno Candel, Jon McKinney, Philipp Singer +4
Large Language Models (LLMs) represent a revolution in AI. However, they also pose many significant risks, such as the presence of biased, private, copyrighted or harmful text. For…