Marvel: A Data-centric Compiler for DNN Operators on Spatial Accelerators

Abstract

The efficiency of a spatial DNN accelerator depends heavily on the compiler and its cost model ability to generate optimized mappings for various operators of DNN models on to the accelerator's compute and memory resources. But, existing cost models lack a formal boundary over the operators for precise and tractable analysis, which poses adaptability challenges for new DNN operators. To address this challenge, we leverage the recently introduced Maestro Data-Centric (MDC) notation. We develop a formal understanding of DNN operators whose mappings can be described in the MDC notation, because any mapping adhering to the notation is always analyzable by the MDC's cost model. Furthermore, we introduce a transformation for translating mappings into the MDC notation for exploring the mapping space. Searching for the optimal mappings is challenging because of the large space of mappings, and this challenge gets exacerbated with new operators and diverse accelerator configurations.To address this challenge, we propose a decoupled off-chip/on-chip approach that decomposes the mapping space into off-chip and on-chip subspaces, and first optimizes the off-chip subspace followed by the on-chip subspace. The motivation for this decomposition is to reduce the size of the search space dramatically and also to prioritize the optimization of off-chip data movement, which is 2-3 orders of magnitude more compared to the on-chip data movement. We implemented our approach in a tool called {\em Marvel}, and another major benefit of our approach is that it is applicable to any DNN operator conformable with the MDC notation.

References in corpus (7)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• DeepPose: Human Pose Estimation via Deep Neural Networks
• End to End Learning for Self-Driving Cars
• MobileNetV2: Inverted Residuals and Linear Bottlenecks
• A Systematic Approach to Blocking Convolutional Neural Networks
• Efficient Processing of Deep Neural Networks: A Tutorial and Survey

Cited by in corpus (7)

• Mind Mappings: Enabling Efficient Algorithm-Accelerator Mapping Space Search
• Tensor Slicing and Optimization for Multicore NPUs
• CoSA: Scheduling by Constrained Optimization for Spatial Accelerators
• Search for Optimal Systolic Arrays: A Comprehensive Automated Exploration Framework and Lessons Learned
• Data-Driven Offline Optimization For Architecting Hardware Accelerators
• RHNAS: Realizable Hardware and Neural Architecture Search
• Union: A Unified HW-SW Co-Design Ecosystem in MLIR for Evaluating Tensor Operations on Spatial Accelerators