3 citations · 7 across the 11 of their papers we have counts for
14 papers
Discrete Diffusion Models are Minimax Optimal Under Sparsity
Luca Blunda, Tyler Farghly, Umut Simsekli +1
Discrete diffusion models have emerged as a leading alternative to autoregressive models for generating text and other categorical data, yet their theoretical foundations remain le…
Distribution Matching Distillation for Continuous Diffusion Language Models
Paul Le Van Kiem, Dario Shariatian, Umut Simsekli +1
Continuous diffusion language models generate all tokens in parallel, yet high-quality generation can still require hundreds of network evaluations (NFEs). We study how distributio…
Tightening the Score Matching Gap for Diffusion Models
Benjamin Dupuis, Tyler Farghly, Maxime Haddouche +2
Diffusion models (DMs) are a state-of-the-art generative method to approximately sample from an unknown distribution. Their training and evaluation primarily rely on an Evidence Lo…
Benign Overfitting Does Not Occur in Diffusion Models
Tyler Farghly, Benjamin Dupuis, Alain Durmus +1
Benign overfitting and double descent have come to shape our understanding of generalization in deep learning, establishing that overfitting is not only compatible with good genera…
Algorithm- and Data-Dependent Generalization Bounds for Diffusion Models
Benjamin Dupuis, Dario Shariatian, Maxime Haddouche +2
Score-based generative models (SGMs) have emerged as one of the most popular classes of generative models. A substantial body of work now exists on the analysis of SGMs, focusing e…
Detecting Benchmark Contamination Through Watermarking
Tom Sander, Pierre Fernandez, Saeed Mahloujifar +2
Benchmark contamination poses a significant challenge to the reliability of Large Language Models (LLMs) evaluations, as it is difficult to assert whether a model has been trained…
Differential privacy guarantees of Markov chain Monte Carlo algorithms
Andrea Bertazzi, Tim Johnston, Gareth O. Roberts +1
This paper aims to provide differential privacy (DP) guarantees for Markov chain Monte Carlo (MCMC) algorithms. In a first part, we establish DP guarantees on samples output by MCM…
Watermark Anything with Localized Messages
Tom Sander, Pierre Fernandez, Alain Durmus +2
Image watermarking methods are not tailored to handle small watermarked areas. This restricts applications in real-world scenarios where parts of the image may come from different…
Piecewise deterministic generative models
Andrea Bertazzi, Dario Shariatian, Umut Simsekli +2
We introduce a novel class of generative models based on piecewise deterministic Markov processes (PDMPs), a family of non-diffusive stochastic processes consisting of deterministi…
Heavy-Tailed Diffusion with Denoising Lévy Probabilistic Models
Dario Shariatian, Umut Simsekli, Alain Durmus
Exploring noise distributions beyond Gaussian in diffusion models remains an open challenge. While Gaussian-based models succeed within a unified SDE framework, recent studies sugg…
Differentially Private Representation Learning via Image Captioning
Tom Sander, Yaodong Yu, Maziar Sanjabi +4
Differentially private (DP) machine learning is considered the gold-standard solution for training a model from sensitive data while still preserving privacy. However, a major barr…
Implicit Bias in Noisy-SGD: With Applications to Differentially Private Training
Tom Sander, Maxime Sylvestre, Alain Durmus
Training Deep Neural Networks (DNNs) with small batches using Stochastic Gradient Descent (SGD) yields superior test performance compared to larger batches. The specific noise stru…
Watermarking Makes Language Models Radioactive
Tom Sander, Pierre Fernandez, Alain Durmus +2
We investigate the radioactivity of text generated by large language models (LLM), i.e. whether it is possible to detect that such synthetic input was used to train a subsequent LL…
Approximate Heavy Tails in Offline (Multi-Pass) Stochastic Gradient Descent
Krunoslav Lehman Pavasovic, Alain Durmus, Umut Simsekli
A recent line of empirical studies has demonstrated that SGD might exhibit a heavy-tailed behavior in practical settings, and the heaviness of the tails might correlate with the ov…