DLVM: A modern compiler infrastructure for deep learning systems
arXiv:1711.03016
Abstract
Deep learning software demands reliability and performance. However, many of the existing deep learning frameworks are software libraries that act as an unsafe DSL in Python and a computation graph interpreter. We present DLVM, a design and implementation of a compiler infrastructure with a linear algebra intermediate representation, algorithmic differentiation by adjoint code generation, domain-specific optimizations and a code generator targeting GPU via LLVM. Designed as a modern compiler infrastructure inspired by LLVM, DLVM is more modular and more generic than existing deep learning compiler frameworks, and supports tensor DSLs with high expressivity. With our prototypical staged DSL embedded in Swift, we argue that the DLVM system enables a form of modular, safe and performant frameworks for deep learning.
References in corpus (1)
Cited by in corpus (17)
- Glow: Graph Lowering Compiler Techniques for Neural Networks
- TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
- Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights
- Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning
- Relay: A New IR for Machine Learning Frameworks
- TensorFlow Eager: A Multi-Stage, Python-Embedded DSL for Machine Learning
- Stripe: Tensor Compilation via the Nested Polyhedral Model
- Data Movement Is All You Need: A Case Study on Optimizing Transformers
- ConfuciuX: Autonomous Hardware Resource Assignment for DNN Accelerators using Reinforcement Learning
- TIRAMISU: A Polyhedral Compiler for Dense and Sparse Deep Learning
- DNNVM : End-to-End Compiler Leveraging Heterogeneous Optimizations on FPGA-based CNN Accelerators
- MNN: A Universal and Efficient Inference Engine
- Auto-Vectorizing TensorFlow Graphs: Jacobians, Auto-Batching And Beyond
- Echo: Compiler-based GPU Memory Footprint Reduction for LSTM RNN Training
- Automatic Horizontal Fusion for GPU Kernels
- Woodpecker-DL: Accelerating Deep Neural Networks via Hardware-Aware Multifaceted Optimizations
- Graph-Based Fuzz Testing for Deep Learning Inference Engine