91 citations · 193 across the 13 of their papers we have counts for
4 papers · 1 filter
Principled Architecture-aware Scaling of Hyperparameters
Wuyang Chen, Junru Wu, Zhangyang Wang +1
Training a high-quality deep neural network requires choosing suitable hyperparameters, which is a non-trivial and expensive process. Current works try to automatically optimize or…
Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis
Wuyang Chen, Wei Huang, Xinyu Gong +2
Advanced deep neural networks (DNNs), designed by either human or AutoML algorithms, are growing increasingly complex. Diverse operations are connected by complicated connectivity…
Auto-scaling Vision Transformers without Training
Wuyang Chen, Wei Huang, Xianzhi Du +3
This work targets automated designing and scaling of Vision Transformers (ViTs). The motivation comes from two pain spots: 1) the lack of efficient and principled methods for desig…
Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metrics
Wuyang Chen, Xinyu Gong, Junru Wu +5
This work targets designing a principled and unified training-free framework for Neural Architecture Search (NAS), with high performance, low cost, and in-depth interpretation. NAS…