4 papers
Learning Where to Steer: Noise-Space Geometry for Efficient Offline Multi-Objective Optimization with Generative Models
Yuan Lu, Esha Singh, Yi-An Ma +1
Offline multi-objective optimization (MOO) seeks solutions with better objective trade-offs using only a fixed dataset, without querying the objectives. Diffusion models trained on…
Residual-Stream Burden Shapes Representation Learning in Diffusion Transformers
Tongtong Liang, Siqi Kou, Ziqiao Xi +6
In diffusion-based generation, a neural network can be trained to predict the clean data, the noise, or the velocity from a noisy input. These prediction targets are interconvertib…
Does Sparse Connectivity Improve Generalization? Convolutional Networks Below the Edge of Stability
Tongtong Liang, Esha Singh, Rahul Parhi +2
Gradient descent on overparameterized neural networks typically operates at the Edge of Stability (EoS), where the largest Hessian eigenvalue hovers around a step-size-dependent th…
Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes
Dan Qiao, Kaiqi Zhang, Esha Singh +2
We study the generalization of two-layer ReLU neural networks in a univariate nonparametric regression problem with noisy labels. This is a problem where kernels (\emph{e.g.} NTK)…