Chameleon: Adaptive Code Optimization for Expedited Deep Neural Network Compilation

Abstract

Achieving faster execution with shorter compilation time can foster further diversity and innovation in neural networks. However, the current paradigm of executing neural networks either relies on hand-optimized libraries, traditional compilation heuristics, or very recently genetic algorithms and other stochastic methods. These methods suffer from frequent costly hardware measurements rendering them not only too time consuming but also suboptimal. As such, we devise a solution that can learn to quickly adapt to a previously unseen design space for code optimization, both accelerating the search and improving the output performance. This solution dubbed Chameleon leverages reinforcement learning whose solution takes fewer steps to converge, and develops an adaptive sampling algorithm that not only focuses on the costly samples (real hardware measurements) on representative points but also uses a domain-knowledge inspired logic to improve the samples itself. Experimentation with real hardware shows that Chameleon provides 4.45x speed up in optimization time over AutoTVM, while also improving inference time of the modern deep networks by 5.6%.

Published as a conference paper at ICLR 2020. arXiv admin note: text overlap with arXiv:1905.12799

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Neural Architecture Search with Reinforcement Learning
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Wide Residual Networks
• AMC: AutoML for Model Compression and Acceleration on Mobile Devices
• Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractions
• Device Placement Optimization with Reinforcement Learning
• Glow: Graph Lowering Compiler Techniques for Neural Networks
• DeepArchitect: Automatically Designing and Training Deep Architectures
• ReLeQ: A Reinforcement Learning Approach for Deep Quantization of Neural Networks

Cited by in corpus (2)

• A Learned Performance Model for Tensor Processing Units
• Ordering Chaos: Memory-Aware Scheduling of Irregularly Wired Neural Networks for Edge Devices