4 papers · 1 filter
From Atoms to Entropy: Optimal Noise Allocation for Diffusion Training in the Convex Regime
Luca Ambrogioni, Giulio Franzese, Alberto Foresti +7
How should a diffusion model decide which noise levels to train on, and how much? Despite the importance of this choice, current noise schedules are based largely on heuristics or…
Towards Diverse and Comprehensive Benchmarks for Mutual Information Estimation
Alberto Foresti, Ivan Butakov, Alexander Tolmachev +3
Mutual information (MI) estimation is a central problem in machine learning and statistics; however, existing benchmarks typically evaluate estimators on simplified, low-dimensiona…
Improved Sampling Schedules for Discrete Diffusion Models
Alberto Foresti, Mustapha Bounoua, Giulio Franzese +2
Discrete diffusion models have emerged as a powerful paradigm for generative modeling on sequence data; however, the information-theoretic principles governing their reverse proces…
INFO-SEDD: Continuous Time Markov Chains as Scalable Information Metrics Estimators
Alberto Foresti, Giulio Franzese, Pietro Michiardi
Information-theoretic quantities play a crucial role in understanding non-linear relationships between random variables and are widely used across scientific disciplines. However,…