TIRAMISU: A Polyhedral Compiler for Dense and Sparse Deep Learning
arXiv:2005.04091
Abstract
In this paper, we demonstrate a compiler that can optimize sparse and recurrent neural networks, both of which are currently outside of the scope of existing neural network compilers (sparse neural networks here stand for networks that can be accelerated with sparse tensor algebra techniques). Our demonstration includes a mapping of sparse and recurrent neural networks to the polyhedral model along with an implementation of our approach in TIRAMISU, our state-of-the-art polyhedral compiler. We evaluate our approach on a set of deep learning benchmarks and compare our results with hand-optimized industrial libraries. Our results show that our approach at least matches Intel MKL-DNN and in some cases outperforms it by 5x (on multicore-CPUs).
References in corpus (10)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Deep Residual Learning for Image Recognition
- AMC: AutoML for Model Compression and Acceleration on Mobile Devices
- To prune, or not to prune: exploring the efficacy of pruning for model compression
- The State of Sparsity in Deep Neural Networks
- Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractions
- Stabilizing the Lottery Ticket Hypothesis
- Faster CNNs with Direct Sparse Convolutions and Guided Pruning
- DLVM: A modern compiler infrastructure for deep learning systems
- Escoin: Efficient Sparse Convolutional Neural Network Inference on GPUs