7 papers
Transformers for dynamical systems learn transfer operators in-context
Anthony Bao, Jeffrey Lai, William Gilpin
Large-scale foundation models for scientific machine learning adapt to physical settings unseen during training, such as zero-shot transfer between turbulent scales. This phenomeno…
Context parroting: A simple but tough-to-beat baseline for foundation models in scientific machine learning
Yuanzhao Zhang, William Gilpin
Recent time-series foundation models exhibit strong abilities to predict physical systems. These abilities include zero-shot forecasting, in which a model forecasts future states o…
Panda: A pretrained forecast model for chaotic dynamics
Jeffrey Lai, Anthony Bao, William Gilpin
Chaotic systems are intrinsically sensitive to small errors, challenging efforts to construct predictive data-driven models of real-world dynamical systems such as fluid flows or n…
Universal Redundancies in Time Series Foundation Models
Anthony Bao, Venkata Hasith Vattikuti, Jeffrey Lai +1
Time Series Foundation Models (TSFMs) leverage extensive pretraining to accurately predict unseen time series during inference, without the need for task-specific fine-tuning. Thro…
Unifying Theories in High-Dimensional Biology: Approaches, Challenges and Opportunities
Marianne Bauer, Akshit Goyal, Sidhartha Goyal +19
Across biological subdisciplines, the last decade has seen an explosion of high-dimensional datasets, including datasets for cells, species, immune systems, neurons and behaviour.…
The cell as a token: high-dimensional geometry in language models and cell embeddings
William Gilpin
Single-cell sequencing technology maps cells to a high-dimensional space encoding their internal activity. Recently-proposed virtual cell models extend this concept, enriching cell…