HCM: Hardware-Aware Complexity Metric for Neural Network Architectures

Abstract

Convolutional Neural Networks (CNNs) have become common in many fields including computer vision, speech recognition, and natural language processing. Although CNN hardware accelerators are already included as part of many SoC architectures, the task of achieving high accuracy on resource-restricted devices is still considered challenging, mainly due to the vast number of design parameters that need to be balanced to achieve an efficient solution. Quantization techniques, when applied to the network parameters, lead to a reduction of power and area and may also change the ratio between communication and computation. As a result, some algorithmic solutions may suffer from lack of memory bandwidth or computational resources and fail to achieve the expected performance due to hardware constraints. Thus, the system designer and the micro-architect need to understand at early development stages the impact of their high-level decisions (e.g., the architecture of the CNN and the amount of bits used to represent its parameters) on the final product (e.g., the expected power saving, area, and accuracy). Unfortunately, existing tools fall short of supporting such decisions. This paper introduces a hardware-aware complexity metric that aims to assist the system designer of the neural network architectures, through the entire project lifetime (especially at its early stages) by predicting the impact of architectural and micro-architectural decisions on the final product. We demonstrate how the proposed metric can help evaluate different design alternatives of neural network models on resource-restricted devices such as real-time embedded systems, and to avoid making design mistakes at early stages.

References in corpus (14)

• Wide Residual Networks
• FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
• Hardware-oriented Approximation of Convolutional Neural Networks
• WRPN: Wide Reduced-Precision Networks
• UNIQ: Uniform Noise Injection for Non-Uniform Quantization of Neural Networks
• Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks
• Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network
• Towards Efficient Training for Neural Network Quantization
• Streaming Architecture for Large-Scale Quantized Neural Networks on an FPGA-Based Dataflow Platform
• Feature Map Transform Coding for Energy-Efficient CNN Inference
• CAT: Compression-Aware Training for bandwidth reduction
• LUTNet: Rethinking Inference in FPGA Soft Logic
• BOPS, Not FLOPS! A New Metric and Roofline Performance Model For Datacenter Computing
• The Pitfall of Evaluating Performance on Emerging AI Accelerators

Cited by in corpus (4)

• A reconfigurable neural network ASIC for detector front-end data compression at the HL-LHC
• Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference
• NICE: Noise Injection and Clamping Estimation for Neural Network Quantization
• Spatio-temporal evolution of resistance state in simulated memristive networks