17 papers
Support Before Frequency in Discrete Diffusion
Adrian Müller, Antoine Gonon, Zebang Shen +2
Discrete diffusion models are increasingly competitive for language modeling, yet it remains unclear how their denoising objectives organize learning. Although these objectives tar…
Select-then-differentiate: Solving Bilevel Optimization with Manifold Lower-level Solution Sets
Saeed Masiha, Zebang Shen, Negar Kiyavash +1
We study optimistic bilevel optimization when the lower-level problem has a non-isolated manifold of minimizers. In this setting, the hyper-objective may be non-differentiable beca…
On the Connectedness of Sublevel Sets in Invex Optimization
Vinzenz Thoma, Zebang Shen, Niao He
Understanding the topology of sublevel sets yields crucial insights into the optimization landscape of non-convex functions. If sublevel sets are connected, local search algorithms…
Manifold Generalization Provably Proceeds Memorization in Diffusion Models
Zebang Shen, Ya-Ping Hsieh, Niao He
Diffusion models often generate novel samples even when the learned score is only \emph{coarse} -- a phenomenon not accounted for by the standard view of diffusion training as dens…
A Schrödinger Eigenfunction Method for Long-Horizon Stochastic Optimal Control
Louis Claeys, Artur Goldman, Zebang Shen +1
High-dimensional stochastic optimal control (SOC) becomes harder with longer planning horizons: existing methods scale linearly in the horizon , with performance often deteriora…
When Scores Learn Geometry: Rate Separations under the Manifold Hypothesis
Xiang Li, Zebang Shen, Ya-Ping Hsieh +1
Score-based methods, such as diffusion models and Bayesian inverse problems, are often interpreted as learning the data distribution in the low-noise limit (). In this wor…