Recurrent Neural Networks with Nonlinear Synapses for Solving Optimization Problems

Abstract

Artificial neural network models typically assume that the synapses are linear. This assumption is not well founded in biological fact, but serves to simplify the analysis and design of artificial neural networks for a variety of tasks, including learning and optimization. In this paper we consider a new form of pulse coupled neural network, where synaptic action is comprised of two components: a nonlinear action which depends on both pre-synaptic and post-synaptic activity levels, and a linear component which is associated with the post-synaptic circuitry alone. We show that interestingly, networks comprising such neurons can be used to efficiently solve difficult optimization problems. Such tasks may arise in the processing of signals in the auditory and visual pathway, but the proposed approach can be used for other applications as well. The assumption of a nonlinear synapse leads to analog neural networks for optimization, which are compact in terms of the number of neurons and interconnections required for a task, in comparison with traditional approaches. It also motivates the design of all-digital neural networks for the same applications. These can be realized in a simple manner on inexpensive programmable digital hardware.

Indexing terms:

• Neural otimization
• Hardware realization
• Pulse coupled neural networks
• Digital neural networks

Additional information

Notes on contributors

Kamlesh Kumar Pathak

Kamlesh Kumar Pathak received the BTech degree in Electronics and Communication Engg from GBPUAT Pantnagar in year 2000, and the MTech degree in Computer Technology from IIT Delhi in 2002. He has been with the System Architecture Group in ST Microelectronics, Noida since 2003. His research interests include VLSI Design and Neural Networks.

Arindam Chakraborthy

Arindam Chakraborty obtained his BE (Electronics & Communications) from the Delhi College of Engineering in 1995 and MTech degree in Integrated Electronics & Circuits from the Indian Institute of Technology, Delhi, in 2000. Since 1995 he has been working as a researcher at the Centre for Development of Telematics (C-DoT), Delhi. His research interests lie in circuits and systems and neural networks. He has been involved with projects in the areas of telecom network synchronization, optical networking, and high speed digital design.