14 citations · 30 across the 10 of their papers we have counts for
3 papers · 1 filter
Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation
Hermann Kumbong, Ishan S. Khare, Iddah Mlauzi +6
Diffusion language models (DLMs) offer parallel sampling for faster generation than autoregressive models, an increasingly important advantage as inference becomes a dominant compu…
Simple linear attention language models balance the recall-throughput tradeoff
Simran Arora, Sabri Eyuboglu, Michael Zhang +6
Recent work has shown that attention-based language models excel at recall, the ability to ground generations in tokens previously seen in context. However, the efficiency of atten…
Zoology: Measuring and Improving Recall in Efficient Language Models
Simran Arora, Sabri Eyuboglu, Aman Timalsina +5
Attention-free language models that combine gating and convolutions are growing in popularity due to their efficiency and increasingly competitive performance. To better understand…