10 papers
A Random Matrix Theory Perspective on the Consistency of Diffusion Models
Binxu Wang, Jacob Zavatone-Veth, Cengiz Pehlevan
Diffusion models trained on different, non-overlapping subsets of a dataset often produce strikingly similar outputs when given the same noise seed. We trace this consistency to a…
Where the Score Lives: A Wavelet View of Diffusion
Emma Finn, Binxu Wang, T. Anderson Keller +1
Score-based generative models have had remarkable success over the last decade in generating a diverse set of visually plausible images. A variety of architectures including CNNs,…
The two clocks and the innovation window: When and how generative models learn rules
Binxu Wang, Emma Lucia Byrnes Finn, Bingbin Liu
Generative models trained on finite data face a fundamental tension: their score-matching or next-token objective converges to the empirical training distribution rather than the p…
Differentiable Faithfulness Alignment for Cross-Model Circuit Transfer
Shun Shao, Binxu Wang, Shay B. Cohen +2
Mechanistic interpretability has made it possible to localize circuits underlying specific behaviors in language models, but existing methods are expensive, model-specific, and dif…
An Analytical Theory of Spectral Bias in the Learning Dynamics of Diffusion Models
Binxu Wang, Cengiz Pehlevan
We develop an analytical framework for understanding how the generated distribution evolves during diffusion model training. Leveraging a Gaussian-equivalence principle, we solve t…
Circuit Mechanisms for Spatial Relation Generation in Diffusion Transformers
Binxu Wang, Jingxuan Fan, Xu Pan
Diffusion Transformers (DiTs) have greatly advanced text-to-image generation, but models still struggle to generate the correct spatial relations between objects as specified in th…