Francesco Locatello

Counterfactual Predictions in Scientific Emulators Without Controlled Experiments

Dingling Yao, Kahaan Gandhi, Valentin Duruisseaux +3

Many scientific questions require reasoning about what was never observed: What if the conditions, interventions, or history had been different? Models can predict accurately on ob…

From Modes to Memories: Characterizing the Scale-Space Dynamics of Diffusion Models

Cristina López Amado, Marco Fumero, Francesco Locatello

Diffusion models are typically viewed as stochastic processes that transform noise into data. We take a complementary perspective: a diffusion model defines a family of determinist…

$λ$ -JEPA Spectral Anti-Collapse Regularization for Self-Supervised Learning

Berker Demirel, Clémentine Dominé, Valentino Maiorca +3

Joint-embedding self-supervised learning typically combines an invariance objective across augmented views with additional mechanisms to prevent representational collapse. These ob…

Thinking at the Right Size: Amortized Distillation Across Post-Trained LLMs

Yan Zhou, Sara Kangaslahti, Jonathan Geuter +4

Practical deployment of large language models (LLMs) requires families of post-trained variants---instruction-tuned, reasoning-tuned, and chat-style models---each at multiple sizes…

Understanding Layer Patching in Model Size Interpolation

Sara Kangaslahti, Jonathan Geuter, Nihal V. Nayak +3

Zero-shot model size interpolation aims to create new models of intermediate target sizes by combining existing models without additional training. Recent work on boomerang distill…

From Tokens to Policy: Causal and Interpretable Heterogeneous Treatment Effects Identification

Riccardo Cadei, Frank Otchere, Nyasha Tirivayi +3

Heterogeneous Treatment Effect (HTE) identification is essential to explain the impact of an intervention and optimize our policies accordingly. Existing approaches trade expressiv…

Assessing Sample Quality in Conditional Generation under Compositional Shift

Berker Demirel, Valentino Maiorca, Marco Fumero +2

Conditional generators provide a natural tool for controllable generation, including settings where the desired condition is a new composition of observed attributes or experimenta…

The Perception-Physics Paradox: Probing Scientific Alignment with TC-Bench

Dingling Yao, Andrea Polesello, Adeel Pervez +2

While Vision Foundation Models (VFMs) excel at predictive tasks on satellite imagery, their performance can arise from visual correlations rather than underlying structural invaria…

Toward Identifiable Sparse Autoencoders

Walter Nelson, Theofanis Karaletsos, Francesco Locatello

Recently, sparse autoencoders (SAEs) have emerged as an attractive tool for interpreting and interacting with representations in practical neural networks. While it is common empir…

Statistical and structural identifiability in representation learning

Walter Nelson, Marco Fumero, Theofanis Karaletsos +1

Representation learning models exhibit a surprising stability in their internal representations. Whereas most prior work treats this stability as a single property, we formalize it…

Addressing Instrument-Outcome Confounding in Mendelian Randomization through Representation Learning

Shimeng Huang, Matthew Robinson, Francesco Locatello

Mendelian Randomization (MR) is a prominent observational epidemiological research method designed to address unobserved confounding when estimating causal effects. However, core a…

A Theory of How Pretraining Shapes Inductive Bias in Fine-Tuning

Nicolas Anguita, Francesco Locatello, Andrew M. Saxe +4

Pretraining and fine-tuning are central stages in modern machine learning systems. In practice, feature learning plays an important role across both stages: deep neural networks le…

Exploratory Causal Inference in SAEnce

Tommaso Mencattini, Riccardo Cadei, Francesco Locatello

Randomized Controlled Trials are one of the pillars of science; nevertheless, they rely on hand-crafted hypotheses and expensive analysis. Such constraints prevent causal effect es…

Boomerang Distillation Enables Zero-Shot Model Size Interpolation

Sara Kangaslahti, Nihal V. Nayak, Jonathan Geuter +3

Large language models (LLMs) are typically deployed under diverse memory and compute constraints. Existing approaches build model families by training each size independently, whic…

Learning Explicit Single-Cell Dynamics Using ODE Representations

Jan-Philipp von Bassewitz, Adeel Pervez, Marco Fumero +3

Modeling the dynamics of cellular differentiation is fundamental to advancing the understanding and treatment of diseases associated with this process, such as cancer. With the rap…

Connecting Neural Models Latent Geometries with Relative Geodesic Representations

Hanlin Yu, Berfin Inal, Georgios Arvanitidis +3

Neural models learn representations of high-dimensional data on low-dimensional manifolds. Multiple factors, including stochasticities in the training process, model architectures,…

Navigating the Latent Space Dynamics of Neural Models

Marco Fumero, Luca Moschella, Emanuele Rodolà +1

Neural networks transform high-dimensional data into compact, structured representations, often modeled as elements of a lower dimensional latent space. In this paper, we present a…

The Third Pillar of Causal Analysis? A Measurement Perspective on Causal Representations

Dingling Yao, Shimeng Huang, Riccardo Cadei +2

Causal reasoning and discovery, two fundamental tasks of causal analysis, often face challenges in applications due to the complexity, noisiness, and high-dimensionality of real-wo…

Mechanistic PDE Networks for Discovery of Governing Equations

Adeel Pervez, Efstratios Gavves, Francesco Locatello

We present Mechanistic PDE Networks -- a model for discovery of governing partial differential equations from data. Mechanistic PDE Networks represent spatiotemporal data as space-…

Prediction-Powered Causal Inferences

Riccardo Cadei, Ilker Demirel, Piersilvio De Bartolomeis +4

In many scientific experiments, the data annotating cost constraints the pace for testing novel hypotheses. Yet, modern machine learning pipelines offer a promising solution, provi…

Identifying General Mechanism Shifts in Linear Causal Representations

Tianyu Chen, Kevin Bello, Francesco Locatello +2

We consider the linear causal representation learning setting where we observe a linear mixing of $d$ unknown latent factors, which follow a linear structural causal model. Recent…

Adjusting Pretrained Backbones for Performativity

Berker Demirel, Lingjing Kong, Kun Zhang +3

With the widespread deployment of deep learning models, they influence their environment in various ways. The induced distribution shifts can lead to unexpected performance degrada…

Scalable Mechanistic Neural Networks for Differential Equations and Machine Learning

Jiale Chen, Dingling Yao, Adeel Pervez +2

We propose Scalable Mechanistic Neural Network (S-MNN), an enhanced neural network framework designed for scientific machine learning applications involving long temporal sequences…