Generative Design of Hardware-aware DNNs
arXiv:2006.03968
Abstract
To efficiently run DNNs on the edge/cloud, many new DNN inference accelerators are being designed and deployed frequently. To enhance the resource efficiency of DNNs, model quantization is a widely-used approach. However, different accelerator/HW has different resources leading to the need for specialized quantization strategy of each HW. Moreover, using the same quantization for every layer may be sub-optimal, increasing the designspace of possible quantization choices. This makes manual-tuning infeasible. Recent work in automatically determining quantization for each layer is driven by optimization methods such as reinforcement learning. However, these approaches need re-training the RL for every new HW platform. We propose a new way for autonomous quantization and HW-aware tuning. We propose a generative model, AQGAN, which takes a target accuracy as the condition and generates a suite of quantization configurations. With the conditional generative model, the user can autonomously generate different configurations with different targets in inference time. Moreover, we propose a simplified HW-tuning flow, which uses the generative model to generate proposals and execute simple selection based on the HW resource budget, whose process is fast and interactive. We evaluate our model on five of the widely-used efficient models on the ImageNet dataset. We compare with existing uniform quantization and state-of-the-art autonomous quantization methods. Our generative model shows competitive achieved accuracy, however, with around two degrees less search cost for each design point. Our generative model shows the generated quantization configuration can lead to less than 3.5% error across all experiments.
References in corpus (18)
- Continuous control with deep reinforcement learning
- Conditional Generative Adversarial Nets
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
- Conditional Image Synthesis With Auxiliary Classifier GANs
- Conditional Image Generation with PixelCNN Decoders
- DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
- ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
- Quantizing deep convolutional networks for efficient inference: A whitepaper
- PACT: Parameterized Clipping Activation for Quantized Neural Networks
- Compressing Neural Networks with the Hashing Trick
- Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
- Semi-Supervised Learning with Generative Adversarial Networks
- Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
- Massively Multitask Networks for Drug Discovery
- Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
- Training deep neural networks with low precision multiplications
- WRPN: Wide Reduced-Precision Networks
- HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks