51 citations · 123 across the 15 of their papers we have counts for
3 papers · 1 filter
Just read twice: closing the recall gap for recurrent language models
Simran Arora, Aman Timalsina, Aaryan Singhal +6
Recurrent large language models that compete with Transformers in language modeling perplexity are emerging at a rapid rate (e.g., Mamba, RWKV). Excitingly, these architectures use…
Counting Clinical Trials: New Evidence on Pharmaceutical Sector Productivity
Maya M. Durvasula, Sabri Eyuboglu, David M. Ritzwoller
We develop a method for assigning high-quality labels to unstructured text. This method is based on fine-tuning an efficient, open-source language model with data extracted from a…
Simple linear attention language models balance the recall-throughput tradeoff
Simran Arora, Sabri Eyuboglu, Michael Zhang +6
Recent work has shown that attention-based language models excel at recall, the ability to ground generations in tokens previously seen in context. However, the efficiency of atten…