16 papers
Multi-Way Representation Alignment
Akshit Achara, Tatiana Gaintseva, Mateo Mahaut +5
The Platonic Representation Hypothesis suggests that independently trained neural networks converge to increasingly similar latent spaces. However, current strategies for mapping t…
Subliminal Clocks: Latent Time Modelling in Diffusion Language Models
Maximo Rulli, Maximo Eduardo Rulli, Thomas Vaitses Fontanari +11
Diffusion Language Models (DLMs) have recently emerged as a promising alternative to autoregressive models. Unlike standard diffusion-based approaches, DLMs are not explicitly cond…
PHALAR: Phasors for Learned Musical Audio Representations
Davide Marincione, Michele Mancusi, Giorgio Strano +4
Stem retrieval, the task of matching missing stems to a given audio submix, is a key challenge currently limited by models that discard temporal information. We introduce PHALAR, a…
Language Models are Injective and Hence Invertible
Giorgos Nikolaou, Tommaso Mencattini, Donato Crisostomi +3
Transformer components such as non-linear activations and normalization are inherently non-injective, suggesting that different inputs could map to the same output and prevent exac…
Humanity's Last Exam
Long Phan, Alice Gatti, Ziwen Han +1144
Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achi…
Membership and Dataset Inference Attacks on Large Audio Generative Models
Jakub Proboszcz, PaweÅ Kochanski, Karol Korszun +5
Generative audio models, based on diffusion and autoregressive architectures, have advanced rapidly in both quality and expressiveness. This progress, however, raises pressing copy…