4 citations · 6 across the 5 of their papers we have counts for
5 papers
Cartridges: Lightweight and general-purpose long context representations via self-study
Sabri Eyuboglu, Ryan Ehrlich, Simran Arora +8
Large language models are often used to answer queries grounded in large text corpora (e.g. codebases, legal documents, or chat histories) by placing the entire corpus in the conte…
Minions: Cost-efficient Collaboration Between On-device and Cloud Language Models
Avanika Narayan, Dan Biderman, Sabri Eyuboglu +4
We investigate an emerging setup in which a small, on-device language model (LM) with access to local data communicates with a frontier, cloud-hosted LM to solve real-world tasks i…
Simple linear attention language models balance the recall-throughput tradeoff
Simran Arora, Sabri Eyuboglu, Michael Zhang +6
Recent work has shown that attention-based language models excel at recall, the ability to ground generations in tokens previously seen in context. However, the efficiency of atten…
Prospector Heads: Generalized Feature Attribution for Large Models & Data
Gautam Machiraju, Alexander Derry, Arjun Desai +6
Feature attribution, the ability to localize regions of the input data that are relevant for classification, is an important capability for ML models in scientific and biomedical d…
Zoology: Measuring and Improving Recall in Efficient Language Models
Simran Arora, Sabri Eyuboglu, Aman Timalsina +5
Attention-free language models that combine gating and convolutions are growing in popularity due to their efficiency and increasingly competitive performance. To better understand…