Interstellar: Using Halide's Scheduling Language to Analyze DNN Accelerators

Abstract

We show that DNN accelerator micro-architectures and their program mappings represent specific choices of loop order and hardware parallelism for computing the seven nested loops of DNNs, which enables us to create a formal taxonomy of all existing dense DNN accelerators. Surprisingly, the loop transformations needed to create these hardware variants can be precisely and concisely represented by Halide's scheduling language. By modifying the Halide compiler to generate hardware, we create a system that can fairly compare these prior accelerators. As long as proper loop blocking schemes are used, and the hardware can support mapping replicated loops, many different hardware dataflows yield similar energy efficiency with good performance. This is because the loop blocking can ensure that most data references stay on-chip with good locality and the processing units have high resource utilization. How resources are allocated, especially in the memory system, has a large impact on energy and performance. By optimizing hardware resource allocation while keeping throughput constant, we achieve up to 4.2X energy improvement for Convolutional Neural Networks (CNNs), 1.6X and 1.8X improvement for Long Short-Term Memories (LSTMs) and multi-layer perceptrons (MLPs), respectively.

Published as a conference paper at ASPLOS 2020

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Sequence to Sequence Learning with Neural Networks
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• EIE: Efficient Inference Engine on Compressed Deep Neural Network
• SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks
• A Systematic Approach to Blocking Convolutional Neural Networks

Cited by in corpus (11)

• DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs
• Pure Tensor Program Rewriting via Access Patterns (Representation Pearl)
• FlexNN: A Dataflow-aware Flexible Deep Learning Accelerator for Energy-Efficient Edge Devices
• How to keep pushing ML accelerator performance? Know your rooflines!
• Stream: Design Space Exploration of Layer-Fused DNNs on Heterogeneous Dataflow Accelerators
• COAC: Cross-layer Optimization of Accelerator Configurability for Efficient CNN Processing
• LOCAL: Low-Complex Mapping Algorithm for Spatial DNN Accelerators
• AXI4MLIR: User-Driven Automatic Host Code Generation for Custom AXI-Based Accelerators
• PACiM: A Sparsity-Centric Hybrid Compute-in-Memory Architecture via Probabilistic Approximation
• Communication Bounds for Convolutional Neural Networks
• ShortcutFusion: From Tensorflow to FPGA-based accelerator with reuse-aware memory allocation for shortcut data