2 citations · 3 across the 10 of their papers we have counts for
11 papers
The Independence Prior of SAEs Fragments Visual Concepts
Tommaso Mencattini, Giorgos Nikolaou, Donato Crisostomi +4
Sparse Autoencoders (SAEs) decompose model activations into sparse combinations of interpretable dictionary atoms. Although SAEs are grounded in the Linear Representation Hypothesi…
Subliminal Clocks: Latent Time Modelling in Diffusion Language Models
Maximo Eduardo Rulli, Thomas Vaitses Fontanari, Simone Petruzzi +9
Diffusion Language Models (DLMs) have recently emerged as a promising alternative to autoregressive models. Unlike standard diffusion-based approaches, DLMs are not explicitly cond…
Steering Vectors are an Adversarial Attack Surface
Abzal Aidakhmetov, Donato Crisostomi, Tommaso Mencattini +3
Activation steering has become a popular way to control Large Language Model (LLM) behavior without fine-tuning. Since the technique is plug-and-play, users share datasets and prec…
How Neural Losses Shape VAE Latents
Giorgio Strano, Luca Cerovaz, Michele Mancusi +2
Modern VAEs are rarely trained with the pointwise likelihood implied by the standard -VAE objective. In practice, pointwise reconstruction is often combined with perceptual and…
The Rate-Distortion-Polysemanticity Tradeoff in SAEs
Tommaso Mencattini, Francesco Montagna, Francesco Locatello
Sparse Autoencoders (SAEs) that can accurately reconstruct their input (minimizing distortion) by making efficient use of few features (minimizing the rate) often fail to learn mon…
Multi-objective Evolutionary Merging Enables Efficient Reasoning Models
Mario Iacobelli, Adrian Robert Minut, Tommaso Mencattini +4
Reasoning models achieve strong performance on complex problems by leveraging long chains of thought, but this deliberate reasoning incurs substantial inference-time cost. The Long…
Language Models are Injective and Hence Invertible
Giorgos Nikolaou, Tommaso Mencattini, Donato Crisostomi +3
Transformer components such as non-linear activations and normalization are inherently non-injective, suggesting that different inputs could map to the same output and prevent exac…
Exploratory Causal Inference in SAEnce
Tommaso Mencattini, Riccardo Cadei, Francesco Locatello
Randomized Controlled Trials are one of the pillars of science; nevertheless, they rely on hand-crafted hypotheses and expensive analysis. Such constraints prevent causal effect es…
Mergenetic: a Simple Evolutionary Model Merging Library
Adrian Robert Minut, Tommaso Mencattini, Andrea Santilli +2
Model merging allows combining the capabilities of existing models into a new one - post hoc, without additional training. This has made it increasingly popular thanks to its low c…
Activation Patching for Interpretable Steering in Music Generation
Simone Facchiano, Giorgio Strano, Donato Crisostomi +4
Understanding how large audio models represent music, and using that understanding to steer generation, is both challenging and underexplored. Inspired by mechanistic interpretabil…
MERGE: Efficient Evolutionary Merging on Consumer-grade GPUs
Tommaso Mencattini, Adrian Robert Minut, Donato Crisostomi +2
Evolutionary model merging enables the creation of high-performing multi-task models but remains computationally prohibitive for consumer hardware. We introduce MERGE, an effic…