31 citations · 123 across the 64 of their papers we have counts for
72 papers
The Independence Prior of SAEs Fragments Visual Concepts
Tommaso Mencattini, Giorgos Nikolaou, Donato Crisostomi +4
Sparse Autoencoders (SAEs) decompose model activations into sparse combinations of interpretable dictionary atoms. Although SAEs are grounded in the Linear Representation Hypothesi…
Emergent Structure in the Marginal Attention Space of Language Models
Valentino Maiorca, Walter Nelson, Francesco Locatello
While representation similarity across independently trained language models is well-documented, how internal mechanics such as attention behave across models remains far less char…
Counterfactual Predictions in Scientific Emulators Without Controlled Experiments
Dingling Yao, Kahaan Gandhi, Valentin Duruisseaux +3
Many scientific questions require reasoning about what was never observed: What if the conditions, interventions, or history had been different? Models can predict accurately on ob…
From Modes to Memories: Characterizing the Scale-Space Dynamics of Diffusion Models
Cristina López Amado, Marco Fumero, Francesco Locatello
Diffusion models are typically viewed as stochastic processes that transform noise into data. We take a complementary perspective: a diffusion model defines a family of determinist…
-JEPA Spectral Anti-Collapse Regularization for Self-Supervised Learning
Berker Demirel, Clémentine Dominé, Valentino Maiorca +3
Joint-embedding self-supervised learning typically combines an invariance objective across augmented views with additional mechanisms to prevent representational collapse. These ob…
LocUS: Head Selection and Subspace Projection for Targeted Activation Steering
Irene Tallini, Lorenzo Basile, Valentino Maiorca +2
Activation steering is a powerful training-free paradigm for controlling large language models at inference time. However, standard approaches estimate a per-layer steering directi…
Thinking at the Right Size: Amortized Distillation Across Post-Trained LLMs
Yan Zhou, Sara Kangaslahti, Jonathan Geuter +4
Practical deployment of large language models (LLMs) requires families of post-trained variants---instruction-tuned, reasoning-tuned, and chat-style models---each at multiple sizes…
Understanding Layer Patching in Model Size Interpolation
Sara Kangaslahti, Jonathan Geuter, Nihal V. Nayak +3
Zero-shot model size interpolation aims to create new models of intermediate target sizes by combining existing models without additional training. Recent work on boomerang distill…
From Tokens to Policy: Causal and Interpretable Heterogeneous Treatment Effects Identification
Riccardo Cadei, Frank Otchere, Nyasha Tirivayi +3
Heterogeneous Treatment Effect (HTE) identification is essential to explain the impact of an intervention and optimize our policies accordingly. Existing approaches trade expressiv…
Assessing Sample Quality in Conditional Generation under Compositional Shift
Berker Demirel, Valentino Maiorca, Marco Fumero +2
Conditional generators provide a natural tool for controllable generation, including settings where the desired condition is a new composition of observed attributes or experimenta…
The Perception-Physics Paradox: Probing Scientific Alignment with TC-Bench
Dingling Yao, Andrea Polesello, Adeel Pervez +2
While Vision Foundation Models (VFMs) excel at predictive tasks on satellite imagery, their performance can arise from visual correlations rather than underlying structural invaria…
Toward Identifiable Sparse Autoencoders
Walter Nelson, Theofanis Karaletsos, Francesco Locatello
Recently, sparse autoencoders (SAEs) have emerged as an attractive tool for interpreting and interacting with representations in practical neural networks. While it is common empir…
Statistical and structural identifiability in representation learning
Walter Nelson, Marco Fumero, Theofanis Karaletsos +1
Representation learning models exhibit a surprising stability in their internal representations. Whereas most prior work treats this stability as a single property, we formalize it…
Addressing Instrument-Outcome Confounding in Mendelian Randomization through Representation Learning
Shimeng Huang, Matthew Robinson, Francesco Locatello
Mendelian Randomization (MR) is a prominent observational epidemiological research method designed to address unobserved confounding when estimating causal effects. However, core a…
A Theory of How Pretraining Shapes Inductive Bias in Fine-Tuning
Nicolas Anguita, Francesco Locatello, Andrew M. Saxe +4
Pretraining and fine-tuning are central stages in modern machine learning systems. In practice, feature learning plays an important role across both stages: deep neural networks le…
MorphGen: Controllable and Morphologically Plausible Generative Cell-Imaging
Berker Demirel, Marco Fumero, Theofanis Karaletsos +1
Simulating in silico cellular responses to interventions is a promising direction to accelerate high-content image-based assays, critical for advancing drug discovery and gene edit…
High-dimensional Analysis of Synthetic Data Selection
Parham Rezaei, Filip Kovacevic, Francesco Locatello +1
Despite the progress in the development of generative models, their usefulness in creating synthetic data that improve prediction performance of classifiers has been put into quest…
Head Pursuit: Probing Attention Specialization in Multimodal Transformers
Lorenzo Basile, Valentino Maiorca, Diego Doimo +2
Language and vision-language models have shown impressive performance across a wide range of tasks, but their internal mechanisms remain only partly understood. In this work, we st…
Exploratory Causal Inference in SAEnce
Tommaso Mencattini, Riccardo Cadei, Francesco Locatello
Randomized Controlled Trials are one of the pillars of science; nevertheless, they rely on hand-crafted hypotheses and expensive analysis. Such constraints prevent causal effect es…
Boomerang Distillation Enables Zero-Shot Model Size Interpolation
Sara Kangaslahti, Nihal V. Nayak, Jonathan Geuter +3
Large language models (LLMs) are typically deployed under diverse memory and compute constraints. Existing approaches build model families by training each size independently, whic…
Learning Explicit Single-Cell Dynamics Using ODE Representations
Jan-Philipp von Bassewitz, Adeel Pervez, Marco Fumero +3
Modeling the dynamics of cellular differentiation is fundamental to advancing the understanding and treatment of diseases associated with this process, such as cancer. With the rap…
Connecting Neural Models Latent Geometries with Relative Geodesic Representations
Hanlin Yu, Berfin Inal, Georgios Arvanitidis +3
Neural models learn representations of high-dimensional data on low-dimensional manifolds. Multiple factors, including stochasticities in the training process, model architectures,…
Navigating the Latent Space Dynamics of Neural Models
Marco Fumero, Luca Moschella, Emanuele Rodolà +1
Neural networks transform high-dimensional data into compact, structured representations, often modeled as elements of a lower dimensional latent space. In this paper, we present a…
The Third Pillar of Causal Analysis? A Measurement Perspective on Causal Representations
Dingling Yao, Shimeng Huang, Riccardo Cadei +2
Causal reasoning and discovery, two fundamental tasks of causal analysis, often face challenges in applications due to the complexity, noisiness, and high-dimensionality of real-wo…