Joint Search of Data Augmentation Policies and Network Architectures

Abstract

The common pipeline of training deep neural networks consists of several building blocks such as data augmentation and network architecture selection. AutoML is a research field that aims at automatically designing those parts, but most methods explore each part independently because it is more challenging to simultaneously search all the parts. In this paper, we propose a joint optimization method for data augmentation policies and network architectures to bring more automation to the design of training pipeline. The core idea of our approach is to make the whole part differentiable. The proposed method combines differentiable methods for augmentation policy search and network architecture search to jointly optimize them in the end-to-end manner. The experimental results show our method achieves competitive or superior performance to the independently searched results.

arXiv commentAAAI 2021 Workshop: Learning Network Architecture during Training

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Neural Architecture Search with Reinforcement Learning
• DARTS: Differentiable Architecture Search
• Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation
• Adversarial AutoAugment
• Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search
• Online Learning Rate Adaptation with Hypergradient Descent
• Stabilizing DARTS with Amended Gradient Estimation on Architectural Parameters
• DSNAS: Direct Neural Architecture Search without Parameter Retraining
• AM-LFS: AutoML for Loss Function Search

Cited by in corpus (2)

• DHA: End-to-End Joint Optimization of Data Augmentation Policy, Hyper-parameter and Architecture
• DAAS: Differentiable Architecture and Augmentation Policy Search