8 papers
Computational and Statistical Guarantees of the \textit{c}-Rectified flow
Leda Wang, Zhehao Xu, Qiang Liu +1
Recently, rectified flow has emerged as a fundamental framework for large-scale image generation, powering state-of-the-art systems such as FLUX.1 and Stable Diffusion 3. Despite i…
Neural Networks Provably Learn Spectral Representations for Group Composition
Jianliang He, Leda Wang, Fengzhuo Zhang +2
Understanding how structured internal structure emerges during neural network training is central to the study of deep learning. We investigate this phenomenon through the group co…
Dynamical mean-field limit and replica-symmetric free energy for the orthogonally-invariant SK model
Zhou Fan, Theodor Misiakiewicz, Leda Wang +1
We study a class of diffusion processes on interacting through a symmetric matrix . When eigenvectors of are Haar-uniform on the orth…
The monotonicity of the Franz-Parisi potential is equivalent with Low-degree MMSE lower bounds
Konstantinos Tsirkas, Leda Wang, Ilias Zadik
Over the last decades, two distinct approaches have been instrumental to our understanding of the computational complexity of statistical estimation. The statistical physics litera…
On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking
Jianliang He, Leda Wang, Siyu Chen +1
We present a comprehensive analysis of how two-layer neural networks learn features to solve the modular addition task. Our work provides a full mechanistic interpretation of the l…
High-dimensional learning dynamics of multi-pass Stochastic Gradient Descent in multi-index models
Zhou Fan, Leda Wang
The learning dynamics of Stochastic Gradient Descent (SGD) are believed to play an important role in influencing the generalization properties of trained models. While the precise…
Understanding and Enhancing Mask-Based Pretraining towards Universal Representations
Mingze Dong, Leda Wang, Yuval Kluger
Mask-based pretraining has become a cornerstone of modern large-scale models across language, vision, and recently biology. Despite its empirical success, its role and limits in le…
Inference in Randomized Least Squares and PCA via Normality of Quadratic Forms
Leda Wang, Zhixiang Zhang, Edgar Dobriban
Randomized algorithms can be used to speed up the analysis of large datasets. In this paper, we develop a unified methodology for statistical inference via randomized sketching or…