7 papers · 1 filter
Distribution Matching Distillation for Continuous Diffusion Language Models
Paul Le Van Kiem, Dario Shariatian, Umut Simsekli +1
Continuous diffusion language models generate all tokens in parallel, yet high-quality generation can still require hundreds of network evaluations (NFEs). We study how distributio…
Uniform Diffusion Models Revisited: Leave-One-Out Denoiser and Absorbing State Reformulation
Samson Gourevitch, Yazid Janati, Dario Shariatian +4
Discrete diffusion models are often trained through clean-data prediction, but the prediction can be used in different ways to define the reverse dynamics. In Masked Diffusion Mode…
Variational Inference for Lévy Process-Driven SDEs via Neural Tilting
Yaman Kindap, Manfred Opper, Benjamin Dupuis +2
Modelling extreme events and heavy-tailed phenomena is central to building reliable predictive systems in domains such as finance, climate science, and safety-critical AI. While Lé…
Latent-Augmented Discrete Diffusion Models
Dario Shariatian, Alain Durmus, Umut Simsekli +1
Discrete diffusion models have emerged as a powerful class of models and a promising route to fast language generation, but practical implementations typically rely on factored rev…
On the Interaction of Compressibility and Adversarial Robustness
Melih Barsbey, Antônio H. Ribeiro, Umut Şimşekli +1
As demands for resource efficiency and safety in modern neural networks intensify, substantial research effort has gone into model compression and adversarial robustness. Yet despi…
The Surprising Agreement Between Convex Optimization Theory and Learning-Rate Scheduling for Large Model Training
Fabian Schaipp, Alexander Hägele, Adrien Taylor +2
We show that learning-rate schedules for large model training behave surprisingly similar to a performance bound from non-smooth convex optimization theory. We provide a bound for…
Topological Generalization Bounds for Discrete-Time Stochastic Optimization Algorithms
Rayna Andreeva, Benjamin Dupuis, Rik Sarkar +2
We present a novel set of rigorous and computationally efficient topology-based complexity notions that exhibit a strong correlation with the generalization gap in modern deep neur…