4 citations · 4 across the 5 of their papers we have counts for
4 papers · 1 filter
Spectral Condition for P under Width-Depth Scaling
Chenyu Zheng, Rongzhen Wang, Xinyu Zhang +1
Generative foundation models are increasingly scaled in both width and depth, posing significant challenges for stable feature learning and reliable hyperparameter (HP) transfer ac…
Scaling Diffusion Transformers Efficiently via P
Chenyu Zheng, Xinyu Zhang, Rongzhen Wang +5
Diffusion Transformers have emerged as the foundation for vision generative models, but their scalability is limited by the high cost of hyperparameter (HP) tuning at large scales.…
A Theory for Conditional Generative Modeling on Multiple Data Sources
Rongzhen Wang, Yan Zhang, Chenyu Zheng +2
The success of large generative models has driven a paradigm shift, leveraging massive multi-source data to enhance model capabilities. However, the interaction among these sources…
On Mesa-Optimization in Autoregressively Trained Transformers: Emergence and Capability
Chenyu Zheng, Wei Huang, Rongzhen Wang +3
Autoregressively trained transformers have brought a profound revolution to the world, especially with their in-context learning (ICL) ability to address downstream tasks. Recently…