Quantized Non-Volatile Nanomagnetic Synapse based Autoencoder for Efficient Unsupervised Network Anomaly Detection

Abstract

In the autoencoder based anomaly detection paradigm, implementing the autoencoder in edge devices capable of learning in real-time is exceedingly challenging due to limited hardware, energy, and computational resources. We show that these limitations can be addressed by designing an autoencoder with low-resolution non-volatile memory-based synapses and employing an effective quantized neural network learning algorithm. We propose a ferromagnetic racetrack with engineered notches hosting a magnetic domain wall (DW) as the autoencoder synapses, where limited state (5-state) synaptic weights are manipulated by spin orbit torque (SOT) current pulses. The performance of anomaly detection of the proposed autoencoder model is evaluated on the NSL-KDD dataset. Limited resolution and DW device stochasticity aware training of the autoencoder is performed, which yields comparable anomaly detection performance to the autoencoder having floating-point precision weights. While the limited number of quantized states and the inherent stochastic nature of DW synaptic weights in nanoscale devices are known to negatively impact the performance, our hardware-aware training algorithm is shown to leverage these imperfect device characteristics to generate an improvement in anomaly detection accuracy (90.98%) compared to accuracy obtained with floating-point trained weights. Furthermore, our DW-based approach demonstrates a remarkable reduction of at least three orders of magnitude in weight updates during training compared to the floating-point approach, implying substantial energy savings for our method. This work could stimulate the development of extremely energy efficient non-volatile multi-state synapse-based processors that can perform real-time training and inference on the edge with unsupervised data.

References in corpus (20)

• The design and verification of Mumax3
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Deep Learning for Anomaly Detection: A Survey
• Mixed-Precision In-Memory Computing
• Early Stage Malware Prediction Using Recurrent Neural Networks
• Hybrid spintronics and straintronics: A magnetic technology for ultra low energy computing and signal processing
• Proposal for an All-Spin Artificial Neural Network: Emulating Neural and Synaptic Functionalities Through Domain Wall Motion in Ferromagnets
• Multiscale Co-Design Analysis of Energy, Latency, Area, and Accuracy of a ReRAM Analog Neural Training Accelerator
• Experimental creation and annihilation of nonvolatile magnetic skyrmions using voltage control of magnetic anisotropy without an external magnetic field
• Mixed-precision deep learning based on computational memory
• Training with Quantization Noise for Extreme Model Compression
• Switching the magnetization of magnetostrictive nanomagnets from single-domain to non-volatile vortex states with a surface acoustic wave
• Synaptic metaplasticity in binarized neural networks
• On-chip learning for domain wall synapse based Fully Connected Neural Network
• Domain Wall-Magnetic Tunnel Junction Spin Orbit Torque Devices and Circuits for In-Memory Computing
• Voltage controlled core reversal of fixed magnetic skyrmions without a magnetic field
• Acoustic Wave Induced FMR Assisted Spin-Torque Switching of Perpendicular MTJs with Anisotropy Variation
• Energy Efficient Learning with Low Resolution Stochastic Domain Wall Synapse Based Deep Neural Networks
• Voltage control of domain walls in magnetic nanowires for energy efficient neuromorphic devices

Cited by in corpus (1)

• Quantized Artificial Neural Networks Implemented with Spintronic Stochastic Computing