40 citations · 62 across the 6 of their papers we have counts for
6 papers
Multi-Token Residual Prediction
Yufeng Xu, Zishuo Bao, Qian Wang +6
Diffusion Language Models (DLMs) generate text by iteratively denoising masked token sequences, offering a tradeoff between parallelism and quality compared to autoregressive model…
SMIR: Efficient Synthetic Data Pipeline To Improve Multi-Image Reasoning
Andrew Li, Rahul Thapa, Rahul Chalamala +3
Vision-Language Models (VLMs) excel at understanding single images, aided by high-quality instruction datasets. However, multi-image reasoning remains underexplored in the open-sou…
RedPajama: an Open Dataset for Training Large Language Models
Maurice Weber, Daniel Fu, Quentin Anthony +16
Large language models are increasingly becoming a cornerstone technology in artificial intelligence, the sciences, and society as a whole, yet the optimal strategies for dataset co…
LoLCATs: On Low-Rank Linearizing of Large Language Models
Michael Zhang, Simran Arora, Rahul Chalamala +5
Recent works show we can linearize large language models (LLMs) -- swapping the quadratic attentions of popular Transformer-based LLMs with subquadratic analogs, such as linear att…
Dragonfly: Multi-Resolution Zoom-In Encoding Enhances Vision-Language Models
Rahul Thapa, Kezhen Chen, Ian Covert +4
Recent advances in vision-language models (VLMs) have demonstrated the advantages of processing images at higher resolutions and utilizing multi-crop features to preserve native re…
LeanDojo: Theorem Proving with Retrieval-Augmented Language Models
Kaiyu Yang, Aidan M. Swope, Alex Gu +6
Large language models (LLMs) have shown promise in proving formal theorems using proof assistants such as Lean. However, existing methods are difficult to reproduce or build on, du…