Stochastic Spintronic Neuron for Hardware Implementation of Neural Networks

Abdolah Amirany, Mohammad Hossein Moaiyeri, Kian Jafari, Masoud Meghdadi

Abstract

The hardware implementation of neural networks has always been of interest to researchers as it can significantly increase the efficiency and application of neural networks due to the distributed nature of Artificial Neural Networks (ANNs) in both memory and computation. Direct implementation of ANNs also offers large gains when scaling the network sizes. Stochastic neurons are among the most significant aspects of machine learning algorithms and are very important in different neural networks. In this paper, a hardware model for the stochastic neuron based on the two-in-one magnetic tunnel junction (TiO-MTJ) in a subcritical current switching regime is proposed. The use of TiO-MTJ has reduced the area of the proposed neuron and eliminated the risk of MTJ read disturbance. Functional evaluation of the proposed model demonstrates that the behavior of the proposed model is comparable to the mathematical description of the stochastic neuron, and it has a negligible error in comparison with the theoretical model. The simulation results of image binarization over 10,000 images indicate that the proposed hardware model has only 0.25% pack signal-to-noise ratio (PSNR) and 0.02% structural similarity (SSIM) variation compared to its software-based counterpart. The results of corners simulations also show the proper performance of the proposed neuron even in the presence of inevitable major process variations.

Keywords

Stochastic Neuron, Spintronic, Magnetic Tunnel Junction (MTJ), Neural Networks, Image Binarization

References

[1] A. Amirany, M. H. Moaiyeri, and K. Jafari, "Nonvolatile Associative Memory Design Based on Spintronic Synapses and CNTFET Neurons," IEEE Transactions on Emerging Topics in Computing, pp. 1-1, 2020.
[2] D. Shin and H.-J. Yoo, "The Heterogeneous Deep Neural Network Processor With a Non-von Neumann Architecture," Proceedings of the IEEE, pp. 1-16, 2019.
[3] S. Basu, R. E. Bryant, G. De Micheli, T. Theis, and L. Whitman, "Nonsilicon, Non-von Neumann Computing—Part I [Scanning the Issue]," Proceedings of the IEEE, vol. 107, no. 1, pp. 11-18, 2019.
[4] A. Amirany and R. Rajaei, "Low Power, and Highly Reliable Single Event Upset Immune Latch for Nanoscale CMOS Technologies," presented at the Electrical Engineering (ICEE), Iranian Conference on, 2018.
[5] M. Krishna Gopi Krishna, A. Roohi, R. Zand, and R. F. DeMara, "Heterogeneous energy-sparing reconfigurable logic: spin-based storage and CNFET-based multiplexing," IET Circuits, Devices & Systems, vol. 11, no. 3, pp. 274-279, 2017.
[6] C. Pan and A. Naeemi, "Non-Boolean Computing Benchmarking for Beyond-CMOS Devices Based on Cellular Neural Network," IEEE Journal on Exploratory Solid-State Computational Devices and Circuits, vol. 2, pp. 36-43, 2016.
[7] A. Amirany, M. H. Moaiyeri, and K. Jafari, "Process-in-Memory Using a Magnetic-Tunnel-Junction Synapse and a Neuron Based on a Carbon Nanotube Field-Effect Transistor," IEEE Magnetics Letters, vol. 10, pp. 1-5, 2019.
[8] A. Amirany, K. Jafari, and M. H. Moaiyeri, "True Random Number Generator for Reliable Hardware Security Modules Based on a Neuromorphic Variation-Tolerant Spintronic Structure," IEEE Transactions on Nanotechnology, pp. 1-1, 2020.
[9] A. Sengupta, Y. Shim, and K. Roy, "Proposal for an All-Spin Artificial Neural Network: Emulating Neural and Synaptic Functionalities Through Domain Wall Motion in Ferromagnets," IEEE Trans Biomed Circuits Syst, vol. 10, no. 6, pp. 1152-1160, Dec 2016.
[10] Z. I. Mannan, S. P. Adhikari, C. Yang, R. K. Budhathoki, H. Kim, and L. Chua, "Memristive Imitation of Synaptic Transmission and Plasticity," IEEE Trans Neural Netw Learn Syst, Feb 11 2019.
[11] Y. Yan et al., "Efficient Reward-Based Structural Plasticity on a SpiNNaker 2 Prototype," IEEE Trans Biomed Circuits Syst, vol. 13, no. 3, pp. 579-591, Jun 2019.
[12] A. Amirany, M. H. Moaiyeri, and K. Jafari, "Bio-Inspired Nonvolatile and Low-Cost Spin-Based 2-Bit per Cell Memory," presented at the 25th International Computer Conference, Computer Society Of Iran, Tehran, Iran, 2020.
[13] A. Amirany, K. Jafari, and M. H. Moaiyeri, "BVA-NQSL: A Bio-inspired Variation Aware Nonvolatile Quaternary Spintronic Latch," IEEE Magnetics Letters, vol. 11, pp. 1-5, 2020.
[14] C. Pan and A. Naeemi, "A Proposal for Energy-Efficient Cellular Neural Network Based on Spintronic Devices," IEEE Transactions on Nanotechnology, vol. 15, no. 5, pp. 820-827, 2016.
[15] K. Gurney, An Introduction to Neural Networks. Taylor \\& Francis, Inc., 1997, p. 288.
[16] S. Haykin, Neural Networks: A Comprehensive Foundation. Prentice Hall PTR, 1994, p. 768.
[17] C. Turchetti, Stochastic Models of Neural Networks. IOS Press, 2004.
[18] A. Mondal and A. Srivastava, "Energy-efficient Design of MTJ-based Neural Networks with Stochastic Computing," ACM Journal on Emerging Technologies in Computing Systems, vol. 16, no. 1, pp. 1-27, 2020.
[19] A. Amirany, K. Jafari, and M. H. Moaiyeri, "High-Performance and Soft Error Immune Spintronic Retention Latch for Highly Reliable Processors," presented at the Electrical Engineering (ICEE), Iranian Conference on, 2020.
[20] Y. Wang, Y. Zhang, E. Y. Deng, J. O. Klein, L. A. B. Naviner, and W. S. Zhao, "Compact model of magnetic tunnel junction with stochastic spin transfer torque switching for reliability analyses," Microelectronics Reliability, vol. 54, no. 9-10, pp. 1774-1778, 2014.
[21] R. Rajaei and A. Amirany, "Nonvolatile Low-Cost Approximate Spintronic Full Adders for Computing in Memory Architectures," IEEE Transactions on Magnetics, vol. 56, no. 4, pp. 1-8, 2020.
[22] R. Rajaei and A. Amirany, "Reliable, High-Performance, and Nonvolatile Hybrid SRAM/MRAM-Based Structures for Reconfigurable Nanoscale Logic Devices," Journal of Nanoelectronics and Optoelectronics, vol. 13, no. 9, pp. 1271-1283, 2018.
[23] S. Huda and A. Sheikholeslami, "A Novel STT-MRAM Cell With Disturbance-Free Read Operation," IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 60, no. 6, pp. 1534-1547, 2013.
[24] J. Grollier, D. Querlioz, K. Y. Camsari, K. Everschor-Sitte, S. Fukami, and M. D. Stiles, "Neuromorphic spintronics," Nature Electronics, 2020.
[25] J. C. Slonczewski, "Conductance and exchange coupling of two ferromagnets separated by a tunneling barrier," Phys Rev B Condens Matter, vol. 39, no. 10, pp. 6995-7002, Apr 1 1989.
[26] Y. Zhang et al., "Compact Modeling of Perpendicular-Anisotropy CoFeB/MgO Magnetic Tunnel Junctions," IEEE Transactions on Electron Devices, vol. 59, no. 3, pp. 819-826, 2012.
[27] A. Amirany, F. Marvi, K. Jafari, and R. Rajaei, "Nonvolatile Spin-Based Radiation Hardened Retention Latch and Flip-Flop," IEEE Transactions on Nanotechnology, vol. 18, pp. 1089-1096, 2019.
[28] Y. Qu, J. Han, B. F. Cockburn, W. Pedrycz, Y. Zhang, and W. Zhao, "A true random number generator based on parallel STT-MTJs," presented at the Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017.
[29] A. Amirany and R. Rajaei, "Fully Nonvolatile and Low Power Full Adder Based on Spin Transfer Torque Magnetic Tunnel Junction With Spin-Hall Effect Assistance," IEEE Transactions on Magnetics, vol. 54, no. 12, pp. 1-7, 2018.
[30] "Predictive Technology Model (PTM) Low Power 45nm Metal Gate / High-K / Strained-Si." http://ptm.asu.edu/modelcard/LP/45nm_LP.pm (accessed 2019)
[31] R. C. Gonzalez, R. E. Woods, and S. L. Eddins, Digital image processing using MATLAB. Pearson Education India, 2004.
[32] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Trans Image Process, vol. 13, no. 4, pp. 600-12, Apr 2004.

Please sign in

The CSI Journal on Computer Science and Engineering

Stochastic Spintronic Neuron for Hardware Implementation of Neural Networks