1.1k citations · 1.2k across the 11 of their papers we have counts for
4 papers · 1 filter
ARC-AGI Without Pretraining
Isaac Liao, Albert Gu
Conventional wisdom in the age of LLMs dictates that solving IQ-test-like visual puzzles from the ARC-AGI-1 benchmark requires capabilities derived from massive pretraining. To cou…
Joint Distillation for Fast Likelihood Evaluation and Sampling in Flow-based Models
Xinyue Ai, Yutong He, Albert Gu +4
Log-likelihood evaluation enables important capabilities in generative models, including model comparison, certain fine-tuning objectives, and many downstream applications. Yet par…
Lyra: An Efficient and Expressive Subquadratic Architecture for Modeling Biological Sequences
Krithik Ramesh, Sameed M. Siddiqui, Albert Gu +2
Deep learning architectures such as convolutional neural networks and Transformers have revolutionized biological sequence modeling, with recent advances driven by scaling up found…
HybriDNA: A Hybrid Transformer-Mamba2 Long-Range DNA Language Model
Mingqian Ma, Guoqing Liu, Chuan Cao +12
Advances in natural language processing and large language models have sparked growing interest in modeling DNA, often referred to as the "language of life". However, DNA modeling…