Conditional Neural Architecture Search

Abstract

Designing resource-efficient Deep Neural Networks (DNNs) is critical to deploy deep learning solutions over edge platforms due to diverse performance, power, and memory budgets. Unfortunately, it is often the case a well-trained ML model does not fit to the constraint of deploying edge platforms, causing a long iteration of model reduction and retraining process. Moreover, a ML model optimized for platform-A often may not be suitable when we deploy it on another platform-B, causing another iteration of model retraining. We propose a conditional neural architecture search method using GAN, which produces feasible ML models for different platforms. We present a new workflow to generate constraint-optimized DNN models. This is the first work of bringing in condition and adversarial technique into Neural Architecture Search domain. We verify the method with regression problems and classification on CIFAR-10. The proposed workflow can successfully generate resource-optimized MLP or CNN-based networks.

References in corpus (16)

• Continuous control with deep reinforcement learning
• Conditional Generative Adversarial Nets
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Conditional Image Generation with PixelCNN Decoders
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• AMC: AutoML for Model Compression and Acceleration on Mobile Devices
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
• Semi-Supervised Learning with Generative Adversarial Networks
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Massively Multitask Networks for Drug Discovery
• Training deep neural networks with low precision multiplications
• MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning
• Understanding Reuse, Performance, and Hardware Cost of DNN Dataflows: A Data-Centric Approach Using MAESTRO