Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
LRCC: Generalizing Low-Rank Compression with Conditional Computation
Thomas Vaitses Fontanari, Maximo Eduardo Rulli, Federico Alvetreti +2
Low-rank compression reduces the cost of pretrained language models by replacing linear transformations with low-rank factorizations. However, conventional methods use a fixed rank…
cs.CL2025
Attention Sinks in Diffusion Language Models
Maximo Eduardo Rulli, Simone Petruzzi, Edoardo Michielon +3
Masked Diffusion Language Models (DLMs) have recently emerged as a promising alternative to traditional Autoregressive Models (ARMs). DLMs employ transformer encoders with bidirect…