Differentiable Architecture Search with Ensemble Gumbel-Softmax
arXiv:1905.01786
Abstract
For network architecture search (NAS), it is crucial but challenging to simultaneously guarantee both effectiveness and efficiency. Towards achieving this goal, we develop a differentiable NAS solution, where the search space includes arbitrary feed-forward network consisting of the predefined number of connections. Benefiting from a proposed ensemble Gumbel-Softmax estimator, our method optimizes both the architecture of a deep network and its parameters in the same round of backward propagation, yielding an end-to-end mechanism of searching network architectures. Extensive experiments on a variety of popular datasets strongly evidence that our method is capable of discovering high-performance architectures, while guaranteeing the requisite efficiency during searching.
References in corpus (11)
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Categorical Reparameterization with Gumbel-Softmax
- DARTS: Differentiable Architecture Search
- Rethinking the Inception Architecture for Computer Vision
- Pointer Sentinel Mixture Models
- ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
- Neural Combinatorial Optimization with Reinforcement Learning
- SNAS: Stochastic Neural Architecture Search
- MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning
- On the State of the Art of Evaluation in Neural Language Models
- Practical Block-wise Neural Network Architecture Generation