Third ArchEdge Workshop: Exploring the Design Space of Efficient Deep Neural Networks
arXiv:2011.10912
Abstract
This paper gives an overview of our ongoing work on the design space exploration of efficient deep neural networks (DNNs). Specifically, we cover two aspects: (1) static architecture design efficiency and (2) dynamic model execution efficiency. For static architecture design, different from existing end-to-end hardware modeling assumptions, we conduct full-stack profiling at the GPU core level to identify better accuracy-latency trade-offs for DNN designs. For dynamic model execution, different from prior work that tackles model redundancy at the DNN-channels level, we explore a new dimension of DNN feature map redundancy to be dynamically traversed at runtime. Last, we highlight several open questions that are poised to draw research attention in the next few years.
Presented in Third ArchEdge Workshop, Co-located with SEC'2020
References in corpus (8)
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
- Neural Architecture Search with Reinforcement Learning
- AMC: AutoML for Model Compression and Acceleration on Mobile Devices
- Dynamic Channel Pruning: Feature Boosting and Suppression
- DNA: Differentiable Network-Accelerator Co-Search
- NASGEM: Neural Architecture Search via Graph Embedding Method
- IOS: Inter-Operator Scheduler for CNN Acceleration
- Towards Latency-aware DNN Optimization with GPU Runtime Analysis and Tail Effect Elimination