133 citations · 570 across the 37 of their papers we have counts for
10 papers · 1 filter
AI Mindmeld: A Latent Interconnect Protocol
Siavash Golkar, Jake Kovalic, Shirley Ho
LLMs can act as scientific agents using tools to consult specialists and run experiments. Yet specialist information must enter their context, generally as text, images, audio or v…
Predicting partially observable dynamical systems via diffusion models with a multiscale inference scheme
Rudy Morel, Francesco Pio Ramunno, Jeff Shen +18
Conditional diffusion models provide a natural framework for probabilistic prediction of dynamical systems and have been successfully applied to fluid dynamics and weather predicti…
Walrus: A Cross-Domain Foundation Model for Continuum Dynamics
Michael McCabe, Payel Mukhopadhyay, Tanya Marwah +22
Foundation models have transformed machine learning for language and vision, but achieving comparable impact in physical simulation remains a challenge. Data heterogeneity and unst…
Joint Embeddings Go Temporal
Sofiane Ennadir, Siavash Golkar, Leopoldo Sarra
Self-supervised learning has seen great success recently in unsupervised representation learning, enabling breakthroughs in natural language and image processing. However, these me…
Contextual Counting: A Mechanistic Study of Transformers on a Quantitative Task
Siavash Golkar, Alberto Bietti, Mariel Pettee +12
Transformers have revolutionized machine learning across diverse domains, yet understanding their behavior remains crucial, particularly in high-stakes applications. This paper int…
Reusability report: Prostate cancer stratification with diverse biologically-informed neural architectures
Christian Pedersen, Tiberiu Tesileanu, Tinghui Wu +4
In Elmarakeby et al., "Biologically informed deep neural network for prostate cancer discovery", a feedforward neural network with biologically informed, sparse connections (P-NET)…
Multiple Physics Pretraining for Physical Surrogate Models
Michael McCabe, Bruno Régaldo-Saint Blancard, Liam Holden Parker +11
We introduce multiple physics pretraining (MPP), an autoregressive task-agnostic pretraining approach for physical surrogate modeling of spatiotemporal systems with transformers. I…
Emergent Structures and Lifetime Structure Evolution in Artificial Neural Networks
Siavash Golkar
Motivated by the flexibility of biological neural networks whose connectivity structure changes significantly during their lifetime, we introduce the Unstructured Recursive Network…
Task-Driven Data Verification via Gradient Descent
Siavash Golkar, Kyunghyun Cho
We introduce a novel algorithm for the detection of possible sample corruption such as mislabeled samples in a training dataset given a small clean validation set. We use a set of…
Continual Learning via Neural Pruning
Siavash Golkar, Michael Kagan, Kyunghyun Cho
We introduce Continual Learning via Neural Pruning (CLNP), a new method aimed at lifelong learning in fixed capacity models based on neuronal model sparsification. In this method,…