7 papers
Introduction to Stochastic Differential Equations for Generative Machine Learning: A Variational Perspective
Ole Winther, Paul Jeha, Sander Dieleman +3
The use of ordinary and stochastic differential equations has led to substantial progress in generative machine learning with applications to, for example, image, video and biomole…
Logit Distance Bounds Representational Similarity
Beatrix M. G. Nielsen, Emanuele Marconato, Luigi Gresele +2
For a broad family of discriminative models that includes autoregressive language models, identifiability results imply that if two models induce the same conditional distributions…
Sparse Shift Autoencoders for Identifying Concepts from Large Language Model Activations
Shruti Joshi, Andrea Dittadi, Sébastien Lachapelle +1
Unsupervised approaches to large language model (LLM) interpretability, such as sparse autoencoders (SAEs), offer a way to decode LLM activations into interpretable and, ideally, c…
Breaking the Likelihood-Quality Trade-off in Diffusion Models by Merging Pretrained Experts
Yasin Esfandiari, Stefan Bauer, Sebastian U. Stich +1
Diffusion models for image generation often exhibit a trade-off between perceptual sample quality and data likelihood: training objectives emphasizing high-noise denoising steps yi…
When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspective
Beatrix M. G. Nielsen, Emanuele Marconato, Andrea Dittadi +1
When and why representations learned by different deep neural networks are similar is an active research topic. We choose to address these questions from the perspective of identif…
Minimum-Excess-Work Guidance
Christopher Kolloff, Tobias Höppe, Emmanouil Angelis +4
We propose a regularization framework inspired by thermodynamic work for guiding pre-trained probability flow generative models (e.g., continuous normalizing flows or diffusion mod…