4 papers
Subliminal Clocks: Latent Time Modelling in Diffusion Language Models
Maximo Rulli, Maximo Eduardo Rulli, Thomas Vaitses Fontanari +11
Diffusion Language Models (DLMs) have recently emerged as a promising alternative to autoregressive models. Unlike standard diffusion-based approaches, DLMs are not explicitly cond…
MASS: MoErging through Adaptive Subspace Selection
Donato Crisostomi, Alessandro Zirilli, Antonio Andrea Gargiulo +5
Model merging has recently emerged as a lightweight alternative to ensembling, combining multiple fine-tuned models into a single set of parameters with no additional training over…
Attention Sinks in Diffusion Language Models
Maximo Eduardo Rulli, Simone Petruzzi, Edoardo Michielon +3
Masked Diffusion Language Models (DLMs) have recently emerged as a promising alternative to traditional Autoregressive Models (ARMs). DLMs employ transformer encoders with bidirect…
Task Singular Vectors: Reducing Task Interference in Model Merging
Antonio Andrea Gargiulo, Donato Crisostomi, Maria Sofia Bucarelli +3
Task Arithmetic has emerged as a simple yet effective method to merge models without additional training. However, by treating entire networks as flat parameter vectors, it overloo…