11 papers
Understanding Layer Patching in Model Size Interpolation
Sara Kangaslahti, Jonathan Geuter, Nihal V. Nayak +3
Zero-shot model size interpolation aims to create new models of intermediate target sizes by combining existing models without additional training. Recent work on boomerang distill…
Assessing Sample Quality in Conditional Generation under Compositional Shift
Berker Demirel, Valentino Maiorca, Marco Fumero +2
Conditional generators provide a natural tool for controllable generation, including settings where the desired condition is a new composition of observed attributes or experimenta…
Navigating the Latent Space Dynamics of Neural Models
Marco Fumero, Luca Moschella, Emanuele Rodolà +1
Neural networks transform high-dimensional data into compact, structured representations, often modeled as elements of a lower dimensional latent space. In this paper, we present a…
Statistical and structural identifiability in representation learning
Walter Nelson, Marco Fumero, Theofanis Karaletsos +1
Representation learning models exhibit a surprising stability in their internal representations. Whereas most prior work treats this stability as a single property, we formalize it…
Learning Explicit Single-Cell Dynamics Using ODE Representations
Jan-Philipp von Bassewitz, Adeel Pervez, Marco Fumero +3
Modeling the dynamics of cellular differentiation is fundamental to advancing the understanding and treatment of diseases associated with this process, such as cancer. With the rap…
Boomerang Distillation Enables Zero-Shot Model Size Interpolation
Sara Kangaslahti, Nihal V. Nayak, Jonathan Geuter +3
Large language models (LLMs) are typically deployed under diverse memory and compute constraints. Existing approaches build model families by training each size independently, whic…