24 papers
Walrus: A Cross-Domain Foundation Model for Continuum Dynamics
Michael McCabe, Payel Mukhopadhyay, Tanya Marwah +22
Foundation models have transformed machine learning for language and vision, but achieving comparable impact in physical simulation remains a challenge. Data heterogeneity and unst…
Protein Design with Agent Rosetta: A Case Study for Specialized Scientific Agents
Jacopo Teneggi, S. M. Bargeen A. Turzo, Tanya Marwah +4
Large language models (LLMs) are capable of emulating reasoning and using tools, creating opportunities for autonomous agents that execute complex scientific tasks. Protein design…
Emergent Transfer of a Physics Foundation Model from Simulation to Laboratory Turbulence
Payel Mukhopadhyay, Stefan S. Nixon, Romain Watteaux +20
Whether physics foundation models can be usefully deployed on laboratory experiments remains an open question for scientific machine learning (ML). We test this question on the Ray…
Assign and Add: A Mechanistic Study of Compositional Arithmetic
Brady Exoo, Alberto Bietti, John Sous
Large language models are able to compose skills in order to perform complex tasks, many of which might not have been seen during training. The details of how exactly this composit…
Multimodal Alignment and Preference Optimization for Zero-Shot Conditional RNA Generation
Roman Klypa, Alberto Bietti, Sergei Grudinin
The design of RNA molecules that interact with specific proteins is a critical challenge in experimental and computational biology. Despite recent progress in natural language mode…
Geometric Factual Recall in Transformers
Shauli Ravfogel, Gilad Yehudai, Joan Bruna +1
How do transformer language models memorize factual associations? A common view casts internal weight matrices as associative memories over pairs of embeddings, requiring parameter…