HTVM: Efficient Neural Network Deployment On Heterogeneous TinyML Platforms

Abstract

Optimal deployment of deep neural networks (DNNs) on state-of-the-art Systems-on-Chips (SoCs) is crucial for tiny machine learning (TinyML) at the edge. The complexity of these SoCs makes deployment non-trivial, as they typically contain multiple heterogeneous compute cores with limited, programmer-managed memory to optimize latency and energy efficiency. We propose HTVM - a compiler that merges TVM with DORY to maximize the utilization of heterogeneous accelerators and minimize data movements. HTVM allows deploying the MLPerf(TM) Tiny suite on DIANA, an SoC with a RISC-V CPU, and digital and analog compute-in-memory AI accelerators, at 120x improved performance over plain TVM deployment.

Presented at DAC2023. Open-source code is available at https://github.com/KULeuven-MICAS/htvm

References in corpus (4)

• A Microprocessor implemented in 65nm CMOS with Configurable and Bit-scalable Accelerator for Programmable In-memory Computing
• DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs
• MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning
• Bring Your Own Codegen to Deep Learning Compiler

Cited by in corpus (1)

• Marsellus: A Heterogeneous RISC-V AI-IoT End-Node SoC with 2-to-8b DNN Acceleration and 30%-Boost Adaptive Body Biasing