1 citations · 1 across the 6 of their papers we have counts for
3 papers · 1 filter
Atomistic Language Models Understand and Generate Materials
Sathya Edamadaka, Krithik Ramesh, Ju Li +1
Atomistic structure and natural language have long been modeled separately, with language models either calling atomistic models as tools or being fine-tuned on lossy textual encod…
Lyra: An Efficient and Expressive Subquadratic Architecture for Modeling Biological Sequences
Krithik Ramesh, Sameed M. Siddiqui, Albert Gu +2
Deep learning architectures such as convolutional neural networks and Transformers have revolutionized biological sequence modeling, with recent advances driven by scaling up found…
LoLCATs: On Low-Rank Linearizing of Large Language Models
Michael Zhang, Simran Arora, Rahul Chalamala +5
Recent works show we can linearize large language models (LLMs) -- swapping the quadratic attentions of popular Transformer-based LLMs with subquadratic analogs, such as linear att…