New Algebraic Activation Function for Multi-Layered Feed Forward Neural Networks

REFERENCES

• S. Goh and D. P. Mandic, “Recurrent neural networks with trainable amplitude of activation functions,” Neural Netw., Vol. 16, no. 8, pp. 1095–1100, May 2003.
   PubMed Web of Science ®Google Scholar
• M. Solazzi and A. Uncini, “Regularising neural networks using flexible multivariate activation function,” Neural Netw., Vol. 17, no. 2, pp. 247–60, Mar. 2004.
   PubMed Web of Science ®Google Scholar
• L. Vecci, F. Piazza, and A. Uncini, “Learning and approximation capabilities of adaptive spline activation function neural networks,” Neural Netw., Vol. 11, no. 2, pp. 259–70, Mar. 1998.
   PubMed Web of Science ®Google Scholar
• J. Han and C. Morag, “The influence of the sigmoid function parameters on the speed of back propagation learning,” in Proceedings of the International Workshop on Artificial Neural Networks: From Natural to Artificial Neural Computation, Spain, 1995, pp. 195–201.
   Google Scholar
• S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed. Singapore: Pearson Education, 1998.
   Google Scholar
• Y. Pao, H, Adaptive Pattern Recognition and Neural Networks, Reading, MA: Addison-Wesley, 1989, pp. 161–70.
   Google Scholar
• D. von Seggern, CRC Standard Curves and Surfaces with Mathematics, 2nd ed. Boca Raton, FL: CRC Press, 2007.
   Google Scholar
• T. M. Mitchell, Machine Learning, Hoboken, NJ: John Wiley & Sons, 1997.
   Google Scholar
• D. Hosmer and S. Lemeshov, Applied Logistic Regression, NewYork: Wiley & Sons, 1989.
   Google Scholar
• J. Dombi, “Membership function as an evaluation,” Fuzzy Sets Syst., Vol. 35, pp. 1–22, 1990, doi: 10.1016/0165-0114(90)90014-W.
   Web of Science ®Google Scholar
• P. Chandra and Y. Singh, “An activation function adapting training algorithm for sigmoidal feedforward networks,” Neurocomput. Lett., Vol. 61, pp. 429–37, Oct. 2004.
   Web of Science ®Google Scholar
• L. Zhou and L. Zhang, “A log-sigmoid Lagrangian neural network for solving nonlinear programming,” in Proceedings of ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, Qingdao, China, Jul. 30–Aug. 1, 2007, pp. 427–31.
   Google Scholar
• G. A. Anastassiou, Intelligent Systems: Approximation by Artificial Neural Networks, Intelligent Systems Reference Library 19. Berlin: Springer-Verlag, 2011.
   Google Scholar
• A. R. Barron, “Universal approximation bounds for superpositions of a sigmoidal function,” IEEE Trans. Inform. Theory, Vol. 39, no. 3, pp. 930–45, May 1993.
   Web of Science ®Google Scholar
• G. Lewicki and G. Marino, “Approximation of functions of finite variation by superpositions of a sigmoidal function,” Appl. Math. Lett., Vol. 17, pp. 1147–52, Jan. 2004.
   Web of Science ®Google Scholar
• Z. Chen and F. Cao, “The approximation operators with sigmoidal functions,” Comput. Math. Appl., Vol. 58, no. 4, pp. 758–65, Aug. 2009.
   Web of Science ®Google Scholar
• Z. Chen and F. Cao, “The construction and approximation of a class of neural networks operators with ramp functions,” J. Comput. Anal. Appl., Vol. 14, no. 1, pp. 101–12, Jan. 2012.
   Web of Science ®Google Scholar
• G. H. L. Cheang, “Approximation with neural networks activated by ramp sigmoids,” J. Approx. Theory, Vol. 162, pp. 1450–65, Aug. 2010.
   Web of Science ®Google Scholar
• D. Costarelli and R. Spigler, “Approximation by series of sigmoidal functions with applications to neural networks,” Annali di Matematica Puraed Appl., Vol. 194, no. 1, pp. 289–306, Feb. 2015.
   Web of Science ®Google Scholar
• D. Costarelli and V. Vinti, “Approximation by max-product neural network operators of Kantorovich type,” Results Math., Vol. 69, no. 3, pp. 505–19, Jun. 2016.
   Web of Science ®Google Scholar
• N. Hahm and B. Hong, “Approximation order to a function in C(R) by superposition of a sigmoidal function,” Appl. Math. Lett., Vol. 15, pp. 591–7, Jul. 2002.
   Web of Science ®Google Scholar
• N. Kyurkchiev and S. Markov, Sigmoid Functions: Some Approximation and Modelling Aspects, Some Moduli in Programming Environment Mathematica, UK: LAP (Lambert Acad. Publ.), 2015.
   Google Scholar
• D. Costarelli and R. Spigler, “Solving Volterra integral equations of the second kind by sigmoidal functions approximation,” J. Int. Equ. Appl., Vol. 25, no. 2, pp. 193–222, Nov. 2013.
   Web of Science ®Google Scholar
• D. Costarelli and R. Spigler, “A collocation method for solving nonlinear Volterra integro-dierential equations of the neutral type by sigmoidal functions,” J. Int. Equ. Appl., Vol. 26, no. 1, pp. 15–52, Nov. 2014.
   Web of Science ®Google Scholar
• D. Costarelli, “Neural network operators: Constructive interpolation of multivariate functions,” Neural Netw., Vol. 67, pp. 28–36, Jul. 2015.
   PubMed Web of Science ®Google Scholar
• D. Elliot, “A better activation function for artificial neural networks, the National Science Foundation,” Institute for Systems Research, Washington, DC, ISR Technical Rep. TR-8, 1993. Available: http://ufnalski.edu.pl/zne/ci_2014/papers/Elliott_TR_93-8.pdf
   Google Scholar
• B. Mendil and K. Benmahammed, “Simple activation functions for neural and fuzzy neural networks,” in Proceedings of IEEE International Symposium on Circuits and Systems, Orlando, FL, May 30–Jun. 2, 1999, pp. 347–50.
   Google Scholar
• J. Sopena, E. Romero, and R. Alqu´ezar, “Neural networks with periodic and monotonic activation functions: a comparative study in classification problems,” in Proceedings of International Conference on Artificial Neural Networks, 1999, pp. 323–8.
   Google Scholar
• W. Duch and N. Jankowski, “Survey of neural transfer functions. Neural Comput. Surveys, Vol. 2, pp. 163–212, 1999.
   Google Scholar
• J. Kamnuzaman and S. Aziz, “A note on activation function in multilayer feedforward learning,” in Proceedings of International Joint Conference on Neural Networks, Honolulu, Hawaii, May 12–17, 2002, pp. 519–23.
   Google Scholar
• G. S. D. S. Gomes and T. B. Ludermir, “Optimization of the weights and asymmetric activation function family of neural network for time series forecasting,” Expert Syst. Appl., Vol. 40, no. 16, pp. 6438–46, Nov. 2013.
   Web of Science ®Google Scholar
• G. Miguez, A. E. Xavier, and N. Maculan, “An evaluation of the bihyperbolic function in the optimization of the backpropagation algorithm,” Int. Trans. Operat. Res., Vol. 21, no. 5, pp. 835–54, Sep. 2014.
   Web of Science ®Google Scholar
• M. Abdulkarim, W. F. W. Ahmad, A. Shafie, and R. Razali, “Performance of multi-layer perceptron neural networks with an exponential decay activation function in airwaves estimation,” Int. J. Adv. Studies Comput. Sci. Eng., Vol. 2, no. 2, pp. 25, 2013.
   Google Scholar
• S. M. Siniscalchi and V. M. Salerno, “Adaptation to new microphones using artificial neural networks with trainable activation functions,” IEEE Trans. Neural Netw. Learn. Syst., Apr. 2016.
   PubMedGoogle Scholar
• M. Riedmiller and H. Braun, “A direct adaptive method for faster back-propagation learning: The RPROP algorithm,” in Proceedings of the International Conference on Neural Networks, San Francisco, San Francisco, CA, Mar. 28–Apr. 1, 1993, pp. 586–91.
   Google Scholar
• W. McCulloch and W. Pitts, “A logical calculus of the ideas immanent in nervous activity,” Bull. Math. Biophys., Vol. 5, pp. 115–33, 1943.
   Google Scholar