Optimally Scheduling CNN Convolutions for Efficient Memory Access

Abstract

Embedded inference engines for convolutional networks must be parsimonious in memory bandwidth and buffer sizing to meet power and cost constraints. We present an analytical memory bandwidth model for loop-nest optimization targeting architectures with application managed buffers. We applied this model to optimize the CNN convolution loop-nest. We show that our model is more accurate than previously published models. Using this model we can identify non-trivial dataflow schedules that result in lowest communication bandwidth given tight local buffering constraints. We show that optimal dataflow schedules are implementable in practice and that our model is accurate with respect to a real implementation; moreover, we introduce an accelerator architecture, named Hardware Convolution Block (HWC), which implements the optimal schedules, and we show it achieves up to 14x memory bandwidth reduction compared to a previously published accelerator with a similar memory interface, but implementing a non-optimal schedule.

References in corpus (8)

• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Deep Learning with Limited Numerical Precision
• A Survey of Model Compression and Acceleration for Deep Neural Networks
• XNOR Neural Engine: a Hardware Accelerator IP for 21.6 fJ/op Binary Neural Network Inference
• Ristretto: Hardware-Oriented Approximation of Convolutional Neural Networks
• The Power of Sparsity in Convolutional Neural Networks
• A Systematic Approach to Blocking Convolutional Neural Networks

Cited by in corpus (6)

• MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning
• DRMap: A Generic DRAM Data Mapping Policy for Energy-Efficient Processing of Convolutional Neural Networks
• ROMANet: Fine-Grained Reuse-Driven Off-Chip Memory Access Management and Data Organization for Deep Neural Network Accelerators
• Improving Memory Utilization in Convolutional Neural Network Accelerators
• Evolving Multi-Resolution Pooling CNN for Monaural Singing Voice Separation
• Enabling Highly Efficient Capsule Networks Processing Through A PIM-Based Architecture Design