Glow: Graph Lowering Compiler Techniques for Neural Networks

Abstract

This paper presents the design of Glow, a machine learning compiler for heterogeneous hardware. It is a pragmatic approach to compilation that enables the generation of highly optimized code for multiple targets. Glow lowers the traditional neural network dataflow graph into a two-phase strongly-typed intermediate representation. The high-level intermediate representation allows the optimizer to perform domain-specific optimizations. The lower-level instruction-based address-only intermediate representation allows the compiler to perform memory-related optimizations, such as instruction scheduling, static memory allocation and copy elimination. At the lowest level, the optimizer performs machine-specific code generation to take advantage of specialized hardware features. Glow features a lowering phase which enables the compiler to support a high number of input operators as well as a large number of hardware targets by eliminating the need to implement all operators on all targets. The lowering phase is designed to reduce the input space and allow new hardware backends to focus on a small number of linear algebra primitives.

References in corpus (3)

• Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractions
• Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning
• DLVM: A modern compiler infrastructure for deep learning systems

Cited by in corpus (45)

• Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better
• GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
• Benchmarking TPU, GPU, and CPU Platforms for Deep Learning
• TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems
• MLIR: A Compiler Infrastructure for the End of Moore's Law
• Relay: A New IR for Machine Learning Frameworks
• Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications
• Optimizing CNN Model Inference on CPUs
• Fusion-Catalyzed Pruning for Optimizing Deep Learning on Intelligent Edge Devices
• On-device Training: A First Overview on Existing Systems
• UNIT: Unifying Tensorized Instruction Compilation
• Dynamic Space-Time Scheduling for GPU Inference
• Nimble: Efficiently Compiling Dynamic Neural Networks for Model Inference
• Data Movement Is All You Need: A Case Study on Optimizing Transformers
• Quantune: Post-training Quantization of Convolutional Neural Networks using Extreme Gradient Boosting for Fast Deployment
• First-Generation Inference Accelerator Deployment at Facebook
• Efficient Execution of Quantized Deep Learning Models: A Compiler Approach
• Chameleon: Adaptive Code Optimization for Expedited Deep Neural Network Compilation
• Tensor Slicing and Optimization for Multicore NPUs
• Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator
• FusionStitching: Boosting Memory Intensive Computations for Deep Learning Workloads
• Cortex: A Compiler for Recursive Deep Learning Models
• High-Performance Deep Learning via a Single Building Block
• Bring Your Own Codegen to Deep Learning Compiler
• Ordering Chaos: Memory-Aware Scheduling of Irregularly Wired Neural Networks for Edge Devices
• LazyTensor: combining eager execution with domain-specific compilers
• Echo: Compiler-based GPU Memory Footprint Reduction for LSTM RNN Training
• Horizontally Fused Training Array: An Effective Hardware Utilization Squeezer for Training Novel Deep Learning Models
• Optimizing DNN Compilation for Distributed Training with Joint OP and Tensor Fusion
• Automatic Horizontal Fusion for GPU Kernels
• Compiling Neural Networks for a Computational Memory Accelerator
• The OoO VLIW JIT Compiler for GPU Inference
• Pre-Quantized Deep Learning Models Codified in ONNX to Enable Hardware/Software Co-Design
• Swift for TensorFlow: A portable, flexible platform for deep learning
• Automated Backend-Aware Post-Training Quantization
• MetaTune: Meta-Learning Based Cost Model for Fast and Efficient Auto-tuning Frameworks
• GEVO: GPU Code Optimization using Evolutionary Computation
• OpTC -- A Toolchain for Deployment of Neural Networks on AURIX TC3xx Microcontrollers
• Dynamic Tensor Rematerialization
• NGEMM: Optimizing GEMM for Deep Learning via Compiler-based Techniques
• Spatial Sharing of GPU for Autotuning DNN models
• Enabling Cross-Domain Communication: How to Bridge the Gap between AI and HW Engineers
• Woodpecker-DL: Accelerating Deep Neural Networks via Hardware-Aware Multifaceted Optimizations
• Transferable Graph Optimizers for ML Compilers
• A Quick Introduction to Functional Verification of Array-Intensive Programs