4 papers
Amortized Maximum Inner Product Search with Learned Support Functions
Theo X. Olausson, João Monteiro, Michal Klein +1
Maximum inner product search (MIPS) is a crucial subroutine in machine learning, requiring the identification of a vector taken within a database (the keys) that best aligns with a…
Stochastic KV Routing: Enabling Adaptive Depth-Wise Cache Sharing
Anastasiia Filippova, David Grangier, Marco Cuturi +1
Serving transformer language models with high throughput requires caching Key-Values (KVs) to avoid redundant computation during autoregressive generation. The memory footprint of…
PairBench: Are Vision-Language Models Reliable at Comparing What They See?
Aarash Feizi, Sai Rajeswar, Adriana Romero-Soriano +4
Understanding how effectively large vision language models (VLMs) compare visual inputs is crucial across numerous applications, yet this fundamental capability remains insufficien…
Performance Control in Early Exiting to Deploy Large Models at the Same Cost of Smaller Ones
Mehrnaz Mofakhami, Reza Bayat, Ioannis Mitliagkas +2
Early Exiting (EE) is a promising technique for speeding up inference by adaptively allocating compute resources to data points based on their difficulty. The approach enables pred…