Improving Differentiable Architecture Search with a Generative Model

Abstract

In differentiable neural architecture search (NAS) algorithms like DARTS, the training set used to update model weight and the validation set used to update model architectures are sampled from the same data distribution. Thus, the uncommon features in the dataset fail to receive enough attention during training. In this paper, instead of introducing more complex NAS algorithms, we explore the idea that adding quality synthesized datasets into training can help the classification model identify its weakness and improve recognition accuracy. We introduce a training strategy called ``Differentiable Architecture Search with a Generative Model(DASGM)." In DASGM, the training set is used to update the classification model weight, while a synthesized dataset is used to train its architecture. The generated images have different distributions from the training set, which can help the classification model learn better features to identify its weakness. We formulate DASGM into a multi-level optimization framework and develop an effective algorithm to solve it. Experiments on CIFAR-10, CIFAR-100, and ImageNet have demonstrated the effectiveness of DASGM. Code will be made available.

References in corpus (11)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Conditional Generative Adversarial Nets
• Neural Architecture Search with Reinforcement Learning
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Self-Attention Generative Adversarial Networks
• DARTS: Differentiable Architecture Search
• PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search
• SNAS: Stochastic Neural Architecture Search
• Understanding and Robustifying Differentiable Architecture Search
• ContraGAN: Contrastive Learning for Conditional Image Generation
• DropNAS: Grouped Operation Dropout for Differentiable Architecture Search