Measurement-driven neural-network training for integrated magnetic tunnel junction arrays

Abstract

The increasing scale of neural networks needed to support more complex applications has led to an increasing requirement for area- and energy-efficient hardware. One route to meeting the budget for these applications is to circumvent the von Neumann bottleneck by performing computation in or near memory. An inevitability of transferring neural networks onto hardware is that non-idealities such as device-to-device variations or poor device yield impact performance. Methods such as hardware-aware training, where substrate non-idealities are incorporated during network training, are one way to recover performance at the cost of solution generality. In this work, we demonstrate inference on hardware neural networks consisting of 20,000 magnetic tunnel junction arrays integrated on a complementary metal-oxide-semiconductor chips that closely resembles market-ready spin transfer-torque magnetoresistive random access memory technology. Using 36 dies, each containing a crossbar array with its own non-idealities, we show that even a small number of defects in physically mapped networks significantly degrades the performance of networks trained without defects and show that, at the cost of generality, hardware-aware training accounting for specific defects on each die can recover to comparable performance with ideal networks. We then demonstrate a robust training method that extends hardware-aware training to statistics-aware training, producing network weights that perform well on most defective dies regardless of their specific defect locations. When evaluated on the 36 physical dies, statistics-aware trained solutions can achieve a mean misclassification error on the MNIST dataset that differs from the software-baseline by only 2 %. This statistics-aware training method could be generalized to networks with many layers that are mapped to hardware suited for industry-ready applications.

17 pages, 9 figures

References in corpus (22)

• Sparse learning of stochastic dynamic equations
• Macrospin Models of Spin Transfer Dynamics
• How to quantify deterministic and random influences on the statistics of the foreign exchange market
• Low-Energy Truly Random Number Generation with Superparamagnetic Tunnel Junctions for Unconventional Computing
• A full-stack view of probabilistic computing with p-bits: devices, architectures and algorithms
• Adaptively time stepping the stochastic Landau-Lifshitz-Gilbert equation at nonzero temperature: implementation and validation in MuMax3
• Demonstration of nanosecond operation in stochastic magnetic tunnel junctions
• Energy-efficient stochastic computing with superparamagnetic tunnel junctions
• Nonlinear stochastic modeling with Langevin regression
• Stochastic Spiking Neural Networks Enabled by Magnetic Tunnel Junctions: From Nontelegraphic to Telegraphic Switching Regimes
• Estimation of Kramers-Moyal coefficients at low sampling rates
• Magnetization dynamics: path-integral formalism for the stochastic Landau-Lifshitz-Gilbert equation
• Entropy-reduced retention times in magnetic memory elements: A case of the Meyer-Neldel Compensation Rule
• Experimental Demonstration of Probabilistic Spin Logic by Magnetic Tunnel Junctions
• Modular Compact Modeling of Magnetic Tunnel Junction Devices
• Numerical integration of the stochastic Landau-Lifshitz-Gilbert equation in generic time-discretisation schemes
• Mutual control of stochastic switching for two electrically coupled superparamagnetic tunnel junctions
• Analysis and data-driven reconstruction of bivariate jump-diffusion processes
• Arbitrary-order finite-time corrections for the Kramers-Moyal operator
• An elementary derivation of first and last return times of 1D random walks
• Heisenberg machines with programmable spin-circuits
• Overcoming device unreliability with continuous learning in a population coding based computing system