Variation-aware Binarized Memristive Networks

Abstract

The quantization of weights to binary states in Deep Neural Networks (DNNs) can replace resource-hungry multiply accumulate operations with simple accumulations. Such Binarized Neural Networks (BNNs) exhibit greatly reduced resource and power requirements. In addition, memristors have been shown as promising synaptic weight elements in DNNs. In this paper, we propose and simulate novel Binarized Memristive Convolutional Neural Network (BMCNN) architectures employing hybrid weight and parameter representations. We train the proposed architectures offline and then map the trained parameters to our binarized memristive devices for inference. To take into account the variations in memristive devices, and to study their effect on the performance, we introduce variations in $R_{ON}$ and $R_{OFF}$. Moreover, we introduce means to mitigate the adverse effect of memristive variations in our proposed networks. Finally, we benchmark our BMCNNs and variation-aware BMCNNs using the MNIST dataset.

arXiv comment4 pages, 3 figures, 3 tables

References in corpus (2)

• Binarized Neural Networks
• Accelerating Deterministic and Stochastic Binarized Neural Networks on FPGAs Using OpenCL

Cited by in corpus (6)

• Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications
• MemTorch: An Open-source Simulation Framework for Memristive Deep Learning Systems
• Seizure Detection and Prediction by Parallel Memristive Convolutional Neural Networks
• Memristive Stochastic Computing for Deep Learning Parameter Optimization
• Improving the Accuracy of Analog-Based In-Memory Computing Accelerators Post-Training
• A Bop and Beyond: A Second Order Optimizer for Binarized Neural Networks