7 papers · 1 filter
Rényi Differential Privacy for Heavy-Tailed SDEs via Fractional Poincaré Inequalities
Benjamin Dupuis, Mert Gürbüzbalaban, Umut Şimşekli +3
Characterizing the differential privacy (DP) of learning algorithms has become a major challenge in recent years. In parallel, many studies suggested investigating the behavior of…
Latent-Augmented Discrete Diffusion Models
Dario Shariatian, Alain Durmus, Umut Simsekli +1
Discrete diffusion models have emerged as a powerful class of models and a promising route to fast language generation, but practical implementations typically rely on factored rev…
On the Interaction of Compressibility and Adversarial Robustness
Melih Barsbey, Antônio H. Ribeiro, Umut Şimşekli +1
As demands for resource efficiency and safety in modern neural networks intensify, substantial research effort has gone into model compression and adversarial robustness. Yet despi…
Algorithm- and Data-Dependent Generalization Bounds for Diffusion Models
Benjamin Dupuis, Dario Shariatian, Maxime Haddouche +2
Score-based generative models (SGMs) have emerged as one of the most popular classes of generative models. A substantial body of work now exists on the analysis of SGMs, focusing e…
Algorithmic Stability of Stochastic Gradient Descent with Momentum under Heavy-Tailed Noise
Thanh Dang, Melih Barsbey, A K M Rokonuzzaman Sonet +3
Understanding the generalization properties of optimization algorithms under heavy-tailed noise has gained growing attention. However, the existing theoretical results mainly focus…
Generalization Bounds for Markov Algorithms through Entropy Flow Computations
Benjamin Dupuis, Maxime Haddouche, George Deligiannidis +1
Many learning algorithms can be represented as Markov processes, and understanding their generalization error is a central topic in learning theory. For specific continuous-time no…
The Surprising Agreement Between Convex Optimization Theory and Learning-Rate Scheduling for Large Model Training
Fabian Schaipp, Alexander Hägele, Adrien Taylor +2
We show that learning-rate schedules for large model training behave surprisingly similar to a performance bound from non-smooth convex optimization theory. We provide a bound for…