DESCNet: Developing Efficient Scratchpad Memories for Capsule Network Hardware

Abstract

Deep Neural Networks (DNNs) have been established as the state-of-the-art algorithm for advanced machine learning applications. Recently proposed by the Google Brain's team, the Capsule Networks (CapsNets) have improved the generalization ability, as compared to DNNs, due to their multi-dimensional capsules and preserving the spatial relationship between different objects. However, they pose significantly high computational and memory requirements, making their energy-efficient inference a challenging task. This paper provides, for the first time, an in-depth analysis to highlight the design and management related challenges for the (on-chip) memories deployed in hardware accelerators executing fast CapsNets inference. To enable an efficient design, we propose an application-specific memory hierarchy, which minimizes the off-chip memory accesses, while efficiently feeding the data to the hardware accelerator. We analyze the corresponding on-chip memory requirements and leverage it to propose a novel methodology to explore different scratchpad memory designs and their energy/area trade-offs. Afterwards, an application-specific power-gating technique is proposed to further reduce the energy consumption, depending upon the utilization across different operations of the CapsNets. Our results for a selected Pareto-optimal solution demonstrate no performance loss and an energy reduction of 79% for the complete accelerator, including computational units and memories, when compared to a state-of-the-art design executing Google's CapsNet model for the MNIST dataset.

arXiv commentAccepted for publication at the IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems

References in corpus (13)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Capsules for Object Segmentation
• Capsule Network Performance on Complex Data
• EIE: Efficient Inference Engine on Compressed Deep Neural Network
• SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks
• CapsNet comparative performance evaluation for image classification
• Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions
• NASCaps: A Framework for Neural Architecture Search to Optimize the Accuracy and Hardware Efficiency of Convolutional Capsule Networks
• CapsAcc: An Efficient Hardware Accelerator for CapsuleNets with Data Reuse
• Q-CapsNets: A Specialized Framework for Quantizing Capsule Networks
• Capsules with Inverted Dot-Product Attention Routing
• ReD-CaNe: A Systematic Methodology for Resilience Analysis and Design of Capsule Networks under Approximations
• Towards Linear Time Neural Machine Translation with Capsule Networks

Cited by in corpus (6)

• Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead
• Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Framework
• Exposing Reliability Degradation and Mitigation in Approximate DNNs under Permanent Faults
• Shifting Capsule Networks from the Cloud to the Deep Edge
• RoHNAS: A Neural Architecture Search Framework with Conjoint Optimization for Adversarial Robustness and Hardware Efficiency of Convolutional and Capsule Networks
• FastCaps: A Design Methodology for Accelerating Capsule Network on Field Programmable Gate Arrays