16 citations · 27 across the 25 of their papers we have counts for
9 papers · 1 filter
Emergent Transfer of a Physics Foundation Model from Simulation to Laboratory Turbulence
Payel Mukhopadhyay, Stefan S. Nixon, Romain Watteaux +20
Whether physics foundation models can be usefully deployed on laboratory experiments remains an open question for scientific machine learning (ML). We test this question on the Ray…
Assign and Add: A Mechanistic Study of Compositional Arithmetic
Brady Exoo, Alberto Bietti, John Sous
Large language models are able to compose skills in order to perform complex tasks, many of which might not have been seen during training. The details of how exactly this composit…
Multimodal Alignment and Preference Optimization for Zero-Shot Conditional RNA Generation
Roman Klypa, Alberto Bietti, Sergei Grudinin
The design of RNA molecules that interact with specific proteins is a critical challenge in experimental and computational biology. Despite recent progress in natural language mode…
Geometric Factual Recall in Transformers
Shauli Ravfogel, Gilad Yehudai, Joan Bruna +1
How do transformer language models memorize factual associations? A common view casts internal weight matrices as associative memories over pairs of embeddings, requiring parameter…
MIMIC: A Generative Multimodal Foundation Model for Biomolecules
Siavash Golkar, Jake Kovalic, Irina Espejo Morales +28
Biological function emerges from coupled constraints across sequence, structure, regulation, evolution, and cellular context, yet most foundation models in biology are trained with…
Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge
Bhavya Vasudeva, Puneesh Deora, Alberto Bietti +2
Transformer-based language models excel at in-context learning (ICL), where they can adapt to new tasks based on contextual examples, without parameter updates. In a specific form…
Learning to Recall with Transformers Beyond Orthogonal Embeddings
Nuri Mert Vural, Alberto Bietti, Mahdi Soltanolkotabi +1
Modern large language models (LLMs) excel at tasks that require storing and retrieving knowledge, such as factual recall and question answering. Transformers are central to this ca…
Representation Learning for Spatiotemporal Physical Systems
Helen Qu, Rudy Morel, Michael McCabe +4
Machine learning approaches to spatiotemporal physical systems have primarily focused on next-frame prediction, with the goal of learning an accurate emulator for the system's evol…
Protein Design with Agent Rosetta: A Case Study for Specialized Scientific Agents
Jacopo Teneggi, S. M. Bargeen A. Turzo, Tanya Marwah +4
Large language models (LLMs) are capable of emulating reasoning and using tools, creating opportunities for autonomous agents that execute complex scientific tasks. Protein design…