Bag of Tricks for Neural Architecture Search
arXiv:2107.03719
Abstract
While neural architecture search methods have been successful in previous years and led to new state-of-the-art performance on various problems, they have also been criticized for being unstable, being highly sensitive with respect to their hyperparameters, and often not performing better than random search. To shed some light on this issue, we discuss some practical considerations that help improve the stability, efficiency and overall performance.
References in corpus (8)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Improved Regularization of Convolutional Neural Networks with Cutout
- Searching for Activation Functions
- Shake-Shake regularization
- NAS-Bench-101: Towards Reproducible Neural Architecture Search
- NAS evaluation is frustratingly hard
- Exploring Randomly Wired Neural Networks for Image Recognition
- Rethinking Architecture Selection in Differentiable NAS