Flexible memristor based neuromorphic system for implementing multi-layer neural network algorithms

References

• Taha TM , Hasan R , Yakopcic C , et al . Exploring the design space of specialized multicore neural processors. IEEE International Joint Conference on Neural Networks (IJCNN); 2013 Aug; Dallas, TX. p. 1–8.
   Google Scholar
• Belhadj B , Zheng AJL , Héliot R , et al . Continuous real-world inputs can open up alternative accelerator designs. ACM/IEEE International Symposium on Computer Architecture (ISCA); 2013 Jun; Tel-Aviv, Israel.
   Google Scholar
• Dubey P. Recognition, mining and synthesis moves computers to the era of tera. Technology@Intel Magazine, 2005 Feb.
   Google Scholar
• Chen T , Chen Y , Duranton M , et al . BenchNN: on the broad potential application scope of hardware neural network accelerators. IEEE International Symposium on Workload Characterization (IISWC); 2012 Nov; San Diego, CA. p. 36–45.
   Google Scholar
• Chua LO . Memristor-the missing circuit element. IEEE Transactions on Circuit Theory. 1971 Jan;18(5):507–519.10.1109/TCT.1971.1083337
   Google Scholar
• Strukov DB , Snider GS , Stewart DR , et al . The missing Memristor found. Nature. 2008 Oct;453:80–83.
   Google Scholar
• Jo SH , Kim K-H , Lu W . High-density crossbar arrays based on a Si Memristive system. Nano Lett. 2009 Jan;9(2):870–874.10.1021/nl8037689
   Google Scholar
• Snider GS . Cortical computing with memristive nanodevices. SciDAC Rev. 2008;Winter:58–65. Available from http://www.scidacreview.org/0804/pdf/hardware.pdf
   Google Scholar
• Vourkas I , Stathis D , Sirakoulis GC . Massively parallel analog computing: Ariadne’s thread was made of memristors. IEEE Trans Emerg Topics Comput. Forthcoming.
   Google Scholar
• Vourkas I , Sirakoulis GC . On the analog computational characteristics of memristive networks. IEEE 20th International Conference on Electronics, Circuits, and Systems (ICECS); 2013 Dec; New Delhi, India. p. 309–312.
   Google Scholar
• Taha TM , Hasan R , Yakopcic C . Memristor crossbar based multicore neuromorphic processors. IEEE International SOCC; Las Vegas, NV; 2014. p. 383–389.
   Google Scholar
• Yakopcic C , Hasan R , Taha TM , et al . Memristor-based neuron circuit and method for applying a learning algorithm in SPICE. IET Electron Lett. 2014 Apr;50(7):492–494.
   Google Scholar
• Yakopcic C , Taha TM . Energy efficient perceptron pattern recognition using segmented memristor crossbar arrays. IEEE International Joint Conference on Neural Networks (IJCNN); Aug 2013; Dallas, TX. p. 1–8.
   Google Scholar
• Yakopcic C , Hasan R , Taha TM . Memristor based neuromorphic circuit for ex-situ training of multi-layer neural network algorithms. IEEE International Joint Conference on Neural Networks (IJCNN); Jul 2015; Killarney, Ireland. p. 1–7.
   Google Scholar
• Soudry D , Di Castro D , Gal A . Memristor-based multilayer neural networks with online gradient descent training. IEEE Trans. Neural Networks Learn Syst. 2015 Oct;26(10):2408–2421.10.1109/TNNLS.2014.2383395
   Google Scholar
• Li B , Wang Y , Wang Y , et al . Training itself: mixed-signal training acceleration for memristor-based neural network. 19th Asia and South Pacific Design Automation Conference (ASP-DAC); Jan 2014; Makuhari, Japan. p. 361–366.
   Google Scholar
• Chabi D , Zhao W , Querlioz D , et al . Robust neural logic block (NLB) based on memristor crossbar array. IEEE/ACM International Symposium on Nanoscale Architectures; Jun 2011; San Diego, CA. p. 137–143.
   Google Scholar
• Zamarreño-Ramos C , Camuñas-Mesa LA , Pérez-Carrasco JA , et al . On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex. Front Neurosci Neuromorph Eng. 2011 Mar;5:1–22. Article 26.
   Google Scholar
• Alibart F , Zamanidoost E , Strukov DB . Pattern classification by memristive crossbar circuits with ex-situ and in situ training. Nat Commun. 2013 Jun;4(2072):1–7.
   Google Scholar
• Hasan R , Taha TM . Enabling back propagation training of memristor crossbar neuromorphic processors. IEEE International Joint Conference on Neural Networks; Jul 2014; Beijing, China. p. 21–28.
   Google Scholar
• Hu M , Li H , Wu Q , et al . Memristor crossbar based hardware realization of BSB recall function. International Joint Conference on Neural Networks (IJCNN); Jun 2012; Brisbane, Australia. p. 1–7.
   Google Scholar
• Lee JH , Likharev KK . Defect-tolerant nanoelectronic pattern classifiers. Int J Circuit Theory Appl. 2007 May;35(3):239–264.10.1002/(ISSN)1097-007X
   Google Scholar
• Chabi D , Querlioz D , Zhao W , et al . Robust learning approach for neuro-inspired nanoscale crossbar architecture. ACM J Emerg Technol Comput Syst. 2014 Jan;10(1):1–20.10.1145/2543749
   Google Scholar
• Querlioz D , Bichler O , Gamrat C . Simulation of a memristor based spiking neural network immune to device variations. IEEE International Joint Conference on Neural Networks; Jul 2011; San Jose, CA. p. 1775–1781.
   Google Scholar
• Hu M , Li H , Chen Y , et al . Memristor crossbar-based neuromorphic computing system: A case study. IEEE Trans Neural Networks Learn Syst. 2014 Oct;25(10):1864–1878.
   Google Scholar
• Starzyk J , Basawaraj A . Memristor crossbar architecture for synchronous neural networks. IEEE Trans Circuit Syst I. 2014 Aug;61:2390–2401.10.1109/TCSI.2014.2304653
   Google Scholar
• Sheridan P , Ma W , Lu W . Pattern recognition with memristor networks. IEEE International Symposium on Circuits and Systems (ISCAS); Jun 2014; Melbourne, Austrailia. p. 1078–1081.
   Google Scholar
• Kim Y , Zhang Y , Li P . A digital neuromorphic vlsi architecture with memristor crossbar synaptic array for machine learning. IEEE International SOC Conference (SOCC); Sep 2012; Niagra Falls, NY. p. 328–333.
   Google Scholar
• Sheri AM , Rafique A , Pedrycz W , et al . Contrastive divergence for memristor-based restricted Boltzmann machine. Eng Appl Artif Intell. 2015 Jan;37:336–342.10.1016/j.engappai.2014.09.013
   Google Scholar
• Kataeva I , Merrikh-Bayat F , Zamanidoost E , et al . Efficient training algorithms for neural networks based on memristive crossbar circuits. IEEE International Joint Conference on Neural Networks (IJCNN); Jul 2015; Killarney, Ireland. p. 1–8.
   Google Scholar
• Schemmel J , Fieres J , Meier K . Wafer-scale integration of analog neural networks. IEEE International Joint Conference on Neural Networks (IJCNN); Jun 2008; Hong Kong, China.
   Google Scholar
• Merolla P , Arthur J , Akopyan F , et al . A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm. IEEE Custom Integrated Circuits Conference (CICC); Sep 2011; San Jose, CA. p. 1–4.
   Google Scholar
• Arthur JV , Merolla PA , Akopyan F , et al . Building block of a programmable neuromorphic substrate: a digital neurosynaptic core. IEEE International Joint Conference on Neural Networks (IJCNN); Jun 2012; Brisbane, Australia. p. 1–8.
   Google Scholar
• Rosenthal E , Greshnikov S , Soudry D , et al . A fully analog memristor-based neural network with online gradient training. IEEE International Symposium on Circuits and Systems (ISCAS); May 2016; Montreal, Canada. p. 1394–1397.
   Google Scholar
• Shim Y , Sengupta A , Roy K . Low-power approximate convolution computing unit with domain-wall motion based ‘spin-memristor’ for image processing applications. 53nd ACM/EDAC/IEEE Design Automation Conference (DAC); 2016 Jun; Austin, TX. p. 1–6.
   Google Scholar
• Yakopcic C , Alom MZ , Taha TM . Memristor crossbar deep network implementation based on a convolutional neural network. IEEE/INNS International Joint Conference on Neural Networks; 2016 Jul; Vancouver Canada; p. 963–970.
   Google Scholar
• Shafiee A , Nag A , Muralimanohar N , et al . ISAAC: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars. ACM/IEEE 43rd Annual International Symposium on Computer Architecture; Jun 2016; Seoul, Korea. p. 14–26.
   Google Scholar
• Chi P , Li S , Xu C , et al . PRIME: a novel processing-in-memory architecture for neural network computation in ReRAM-based main memory. ACM/IEEE 43rd Annual International Symposium on Computer Architecture; 2016 Jun; Seoul, Korea. p. 27–39.
   Google Scholar
• Howard G , Bull L , de Lacy Costello B , et al . Evolving spiking networks with variable resistive memories. Evol Comput. 2014 Spring;22(1):79–103.10.1162/EVCO_a_00103
   Google Scholar
• Howard G , Gale E , Bull L , et al . Towards evolving spiking networks with memristive synapses. 2011 IEEE Symposium on Artificial Life (ALIFE); 2011 Apr; Paris, France. p. 1–8.
   Google Scholar
• Howard G , Gale E , Bull L , et al . Evolution of plastic learning in spiking networks via memristive connections. IEEE Trans Evol Comput. 2012 Oct;16(5):711–729.
   Google Scholar
• Gale E , de Lacy Costello B , Adamatzky A . Filamentary extension of the mem-con theory of memristance and its application to titanium dioxide sol-gel memristors. IEEE International Conference on Electronics Design, Systems and Applications (ICEDSA); 2012 Nov; Kuala Lumpur, Malaysia. p. 86–91.
   Google Scholar
• Amirsoleimani A , Ahmadi M , Ahmadi A , et al . Brain-inspired pattern classification with memristive neural network using the Hodgkin-Huxley neuron. IEEE International Conference on Electronics, Circuits and Systems (ICECS); 2016 Dec; Monte Carlo, Monaco.
   Google Scholar
• Georgilas I , Gale E , Adamatzky A , et al . UAV horizon tracking using memristors and cellular automata visual processing. TAROS 2013, The 14th Conference Towards Autonomous Robotic Systems; 2013 August 28–30; St Anne’s College, Oxford, UK. p. 1–12.
   Google Scholar
• Hu X , Feng G , Duan S , et al . A memristive multilayer cellular neural network with applications to image processing. IEEE Trans Neural Networks Learn Syst. Forthcoming.
   Google Scholar
• Sheridan PM , Du C , Lu WD . Feature extraction using memristor networks. IEEE Transactions on Neural Networks and Learning Systems. 2016 Nov;27(11):2327–2336.
   Google Scholar
• Khalid M , Singh J . Memristor crossbar-based pattern recognition circuit using perceptron learning rule. IEEE International Symposium on Nanoelectronic and Information Systems (iNIS); 2016 Dec; Gwalior, India. p. 236–239.
   Google Scholar
• Chowdhury A , Sarkar M , Arka AI , et al . Associative memory algorithm for visual pattern recognition with memristor array and CMOS neuron. 9th International Conference on Electrical and Computer Engineering (ICECE); 2016 Dec; Dhaka, Bangladesh. p. 42–45.
   Google Scholar
• Pershin YV , Di Ventra M . Memcomputing Implementation of Ant Colony Optimization. Neural Process Lett. 2016;44(1):265–277.10.1007/s11063-016-9497-y
   Google Scholar
• Liu C , Yang Q , Zhang C , et al . A memristor-based neuromorphic engine with a current sensing scheme for artificial neural network applications. 22nd Asia and South Pacific Design Automation Conference (ASP-DAC); 2017 Jan; Chiba/Tokyo, Japan. p. 647–652.
   Google Scholar
• Danilin SN , Shchanikov SA , Panteleev SV . Determining operation tolerances of memristor-based artificial neural networks. International Conference on Engineering and Telecommunication (EnT); 2016 Nov; Moscow, Russia. p. 34–38.
   Google Scholar
• Xie G , Liu G , Zhang S. Expression of emotion using a system combined artificial neural network and memristor-based crossbar array. 35th Chinese Control Conference (CCC); 2016 Jul; Chengdu, China. p. 9837–9841.
   Google Scholar
• Vourkas I , Abusleme A , Ntinas V , et al . A digital memristor emulator for FPGA-based artificial neural networks. 1st IEEE International Verification and Security Workshop (IVSW); 2016 Jul; Catalunya, Spain. p. 54–57.
   Google Scholar
• Yi W , Perner F , Qureshi MS , et al . Feedback write scheme for memristive switching devices. Appl Phys A. 2011;102:973–982. DOI:10.1007/s00339-011-6279-2
   Google Scholar
• Rafique MA , Lee BG , Jeon M . Hybrid neuromorphic system for automatic speech recognition. Electron Lett. 2016 Aug;52(17):1428–1430.10.1049/el.2016.0975
   Google Scholar
• Yakopcic C , Hasan R , Taha TM , et al . SPICE analysis of dense memristor crossbars for low power neuromorphic processor designs. IEEE National Aerospace and Electronics Conference; 2015 Jun; Dayton, OH. p. 305–311.
   Google Scholar
• Yu S , Wu Y , Wong H-SP . Investigating the switching dynamics and multilevel capability of bipolar metal oxide resistive switching memory. Appl Phys Lett. 2011;98:103514.10.1063/1.3564883
   Google Scholar
• Alibart F , Gao L , Hoskins B , et al . High-precision tuning of state for memristive devices by adaptable variation-tolerant algorithm. Nanotechnology. 2012 Jan;23:1–7. Art. 075201.
   Google Scholar
• Gada EF , Atiyab AF , Shaheenc S , et al . A new algorithm for learning in piecewise-linear neural networks. Neural Networks. 2000 Jun;13(4–5):484–505.
   Google Scholar
• Yakopcic C , Alom Z , Taha T Extremely parallel memristor crossbar architecture for convolutional neural network implementation. IEEE International Joint Conference on Neural Networks; 2017 May; Anchorage, AK. Forthcoming.
   Google Scholar
• Chang T , Jo SH , Kim KH , et al . Synaptic behaviors and modeling of a metal oxide memristive device. Appl Phys A. 2011 Mar;102(4):857–863.10.1007/s00339-011-6296-1
   Google Scholar
• Oblea AS , Timilsina A , Moore D , et al . Silver chalcogenide based memristor devices. IEEE International Joint Conference on Neural Networks; 2010 Oct; Barcelona, Spain. p. 1–3.
   Google Scholar
• Jo SH , Chang T , Ebong I , et al . Nanoscale memristor device as synapse in neuromorphic systems. Nano Letters. 2010 Mar;10:1297–1301.10.1021/nl904092h
   Google Scholar
• Hu M , Wang Y , Wen W , et al . Leveraging stochastic memristor devices in neuromorphic hardware systems. IEEE J Emerg Select Topics Circuit Syst. 2016 Jun;6(2):235–246.10.1109/JETCAS.2016.2547780
   Google Scholar
• Yakopcic C , Taha TM . Determining optimal switching speed for memristors in a neuromorphic system. Electron Lett. 2015 Oct;51(21):1637–1639.
   Google Scholar
• Lu W , Kim K-H , Chang T , et al . Two-terminal resistive switches (memristors) for memory and logic applications. 16th Asia and South Pacific Design Automation Conference; 2011 Jan; Yokohama, Japan. p. 217–223.
   Google Scholar
• Yakopcic C , Taha TM , Subramanyam G , et al . A memristor device model. IEEE Electron Device Lett. 2011 Oct;30(10):1436–1438.
   Google Scholar
• Yakopcic C , Taha TM , Subramanyam G , et al . Generalized memristive device SPICE model and its application in circuit design. IEEE Trans Comput-Aided Des Integr Circuits Syst. 2013 Aug;32(8):1201–1214.10.1109/TCAD.2013.2252057
   Google Scholar
• Yakopcic C , Taha TM , Subramanyam G , et al . Memristor SPICE model and crossbar simulation with nanosecond switching time. IEEE International Joint Conference on Neural Networks (IJCNN); August 2013; Dallas, TX.
   Google Scholar
• Yakopcic C , Taha TM , Subramanyam G , et al . Impact of memristor switching noise in a neuromorphic crossbar. IEEE National Aerospace and Electronics Conference; 2015 Jun; Dayton, OH. p. 320–326.
   Google Scholar
• Yakopcic C , Hasan R , Taha TM . Hybrid crossbar architecture for a memristor based cache. Microelectron J. 2015 Nov;46(11):1020–1032.
   Google Scholar
• Hinton G . Available from http://www.cs.toronto.edu/~hinton/MatlabForSciencePaper.html
   Google Scholar
• LeCun Y , Cortes C , Burges CJC . Available from http://yann.lecun.com/exdb/mnist/
   Google Scholar