ERANN: An Algorithm to Extract Symbolic Rules from Trained Artificial Neural Networks

References

• K Saito, and R Nakano, “Medical diagnosis expert system based on PDP model,” in Proceedings of IEEE International Conference on Neutal Networks, New York: IEEE Press; pp. 1255–62. 1988.
   Google Scholar
• R Setiono, W K Leow, and J M Zurada, “Extraction of rules from artificial neural networks for nonlinear regression,” IEEE Transactions on Neural Networks, Vol. 13, pp. 564–77, 2002.
   Google Scholar
• B Baesens, R Setiono, C Mues, and J Vanthienen, “Using neural network rule extraction and decision tables for credit-risk evaluation,” J Management Science, Vol. 49, no. 3, pp. 312–29, 2003.
   Google Scholar
• T Hou, C Su, and H Chang, “Using neural networks and immune algorithms to find the optimal parameters for an IC wire bonding process,” Expert System with Applications, Vol. 34, pp. 427–36, 2008.
   Google Scholar
• T Q Huynh, and J A Reggia, “Guiding hidden layer representations for improved rule extraction from neural networks,” IEEE Transactions on Neural Networks, Vol. 22, no. 2, pp. 264–75, 2011.
   Google Scholar
• R Andrews, J Diederich, and A B Tickle, “Survey and critique of techniques for extracting rules from trained artificial neural networks,” Knowledge Based System, Vol. 8, no. 6, pp. 373–89, 1995.
   Google Scholar
• S H Huang, and H Xing, “Extract intelligible and concise fuzzy rules from neural networks,” Fuzzy Sets and Systems, Vol. 132, no. 2, pp. 233–43, 2002.
   Google Scholar
• A Darbari, “Rule extraction from trained ANN: A survey,” Technical Report, Department of Computer Science, Dresden University of Technology, Dresden, Germany, 2000.
   Google Scholar
• Z H Zhou, “Rule extraction: Using neural networks or for neural networks?” Journal of Computer Science and Technology, Vol. 19, no. 2, pp. 249–53, 2004.
   Google Scholar
• H Jacobsson, “Rule extraction from recurrent neural networks: A taxonomy and review,” Neural Computation, Vol. 17, no. 6, pp. 1223–63, 2005.
   Google Scholar
• E R Hruschka, and N F Ebecken, “Extracting rules from multilayer perceptrons in classification problems: A clustering-based approach,” Neurocomputing, Vol. 70, pp. 384–97, 2006.
   Google Scholar
• S Bader, S Holldobler, and V Mayer-Eichberger, “Extracting propositional rules from feed-forward neural networks: A new decompositional approach,” in Proceedings of 3rd International Workshop on Neural-Symbolic Learning and Reasoning, pp. 1–6, 2007.
   Google Scholar
• R Setiono, B Baesens, and C Mues, “Recursive neural network rule extraction for data with mixed attributes,” IEEE Transactions on Neural Networks, Vol. 19, no. 2, pp. 299–307, 2008.
   Google Scholar
• R Nayak, “Generating rules with predicates, terms and variables from the pruned neural networks,” Neural Network, Vol. 22, no. 4, pp. 405–14, May 2009.
   Google Scholar
• J Guerreiro, and D Trigueiros, “A unified approach to the extraction of rules from artificial neural networks and support vector machines,” in Proceedings of International Conference Advanced Data Mining and Applications, ADMA Part II, LNCS 6441, pp. 34–42, 2010.
   Google Scholar
• J Chorowski, and J M Zurada, “Extracting rules from neural networks as decision diagrams,” to appear in the IEEE Transactions on Neural Networks, 2011.
   Google Scholar
• I Taha, and J Ghosh, “Three techniques for extracting rules from feedforward networks,” Intelligent Engineering Systems Through Artificial Neural Networks, vol. 6, pp. 23–8, 1996.
   Google Scholar
• H Tsukimoto, “Extracting rules from trained neural networks,” IEEE Transactions on Neural Networks, Vol. 11, no. 2, pp. 377–89, Mar. 2000.
   Google Scholar
• E W Saad, and D C II Wunsch, “Neural network explanation using inversion,” Neural Networks, Vol. 20, pp. 78–93, 2007.
   Google Scholar
• R Setione, and H Liu, “Neurolinear: From neural networks to oblique decision rules,” Neurocomputing, Vol. 17, no. 1, pp. 1–24, 1997.
   Google Scholar
• A S Garcez d’Avila, K Broda, and D M Gabbay, “Symbolic knowledge from trained neural networks: A sound approach,” Artificial. Intelligence, vol.125, no. 1, pp. 155–207, 2001.
   Google Scholar
• E Keedwell, A Narayanan, and D Savic, “Creating rules from trained neural networks using genetic algorithms” International Journal of Computers Systems and Signals, vol. 1, no. 1, pp. 30–42, 2000.
   Google Scholar
• G G Towell, and J W Shavlik, “Extracting refined rules from knowledge-based system neural networks,” Machine Learning, Vol. 13, pp. 71–101, 1993.
   Google Scholar
• L Fu, “Rule learning by searching on adapted nets,” in Proceedings of the 9th National Conference on Artificial Intelligence, The MIT Press, Menlo Park, CA, pp. 590–5, 1991.
   Google Scholar
• G G Towell, and J W Shavlik, “Knowledge-based artificial neural networks,” Artificial Intelligence, Vol. 70, pp. 119–65, 1994.
   Google Scholar
• M W Craven, and J W Shavlik, “Using sampling and queries to extract rules from trained neural networks,” in Proceedings of the 11th International Conference on Machine Learning, Morgan and Kaufmann, San Mateo, CA, 1994.
   Google Scholar
• H Liu, and S T Tan, “X2R: A fast rule generator,” in Proceedings of IEEE International Conference on Systems, Man and Cybernetics, Vancouver, CA, 1995.
   Google Scholar
• R Setiono, and H Liu, “Understanding neural networks via rule extraction,” in Proceedings of the 14th International Joint Conference on Artificial Intelligence, 1995, pp. 480–5.
   Google Scholar
• R Setiono, and H Liu, “Symbolic presentation of neural networks,” IEEE Computer, pp. 71–77, 1996.
   Google Scholar
• R Setiono, and W K Leow, “FERNN: An algorithm for fast extraction of rules from neural networks,” Applied Intelligence, Vol. 12, pp. 15–25, 2000.
   Google Scholar
• S M Kamruzzaman, and M M Islam, “An algorithm to extract rules from artificial neural networks for medical diagnosis problems”, International Journal of Information Technology, Vol. 12, no. 8, pp. 41–59, 2006.
   Google Scholar
• T A Etchells, and P J Lisboa, “Orthogonal search-based rule extraction (OSRE) for trained neural networks: A practical and efficient approach,” IEEE Transactions on Neural Networks, Vol. 17, no. 2, pp. 374–84, Mar. 2006.
   Google Scholar
• Y Jin, and B Sendhoff, “Pareto-based multiobjective machine learning: An overview and case studies,” IEEE Transactions on System Man and Cybernetics.-Part C: Applications and Reviews, Vol. 38, no. 3, pp. 397–415, 2008.
   Google Scholar
• H Kahramanli, and N Allahverdi, “Rule extraction from trained adaptive neural networks using artificial immune systems,” Expert Systems with Applications, Vol. 36, pp. 1513–22, 2009.
   Google Scholar
• T Y Kwok, and D Y Yeung, “Constructive algorithms for structured learning in feedforward neural networks for regression problems,” IEEE Transactions on Neural Networks, Vol. 8, pp. 630–45, 1997.
   Google Scholar
• M M Islam, X Yao, and K Murase, “A constructive algorithm for training cooperative neural network ensembles,” IEEE Transactions on Neural Networks, Vol. 14, pp. 820–34, 2003.
   Google Scholar
• R Parekh, J Yang, and V Honavar, “Constructive neural network learning algorithms for pattern classification,” IEEE Transactions on Neural Networks, Vol. 11, pp. 436–52, 2000.
   Google Scholar
• T Ash, “Dynamic node creation in backpropagation networks,” Connection Science, Vol. 1, pp. 365–75, 1989.
   Google Scholar
• S E Fahlman, and C Lebiere, “The cascade-correlation learning architecture,” Advances in Neural Information Processing System, In: D S Touretzky, Editor. San Mateo, CA: Morgan Kaufmann; pp. 524–32, 1990.
   Google Scholar
• R Setiono, and L C Hui, “Use of quasi-Newton method in a feedforward neural network construction algorithm,” IEEE Transactions on Neural Networks, Vol. 6, pp. 273–7, 1995.
   Google Scholar
• M M Islam, M A Akhand, M A Rahman, and K Murase, “Weight freezing to reduce training time in designing artificial neural networks,” in Proceedings of 5th International Conference on Computer and Information Technology, EWU, Dhaka, pp. 132–6,2002.
   Google Scholar
• M M Islam, and K Murase, “A new algorithm to design compact two hidden-layer artificial neural networks,” Neural Networks, Vol. 14, pp. 1265–78, 2001.
   Google Scholar
• J Sietsma, and RJ Dow, “Neural net pruning-why and how?,” in Proceedings of IEEE International Conference on Neural Networks, Vol. 1, pp. 325–33, 1988.
   Google Scholar
• A V Ooyen, and B Nienhuis, “Improving the convergence of backpropagation algorithm,” Neural Networks, Vol. 5, pp. 465–71, 1992.
   Google Scholar
• R Reed, “Pruning algorithms-A survey,” IEEE Transactions on Neural Networks, Vol. 4, pp. 740–7, 1993.
   Google Scholar
• J Han, and M Kamber, “Data Mining: Concepts and Techniques,” CA: Morgan Kaufmann Publisher: 2001.
   Google Scholar
• L Kaufman, and P J Rousseeuw, “Finding Groups in Data: An Introduction to Cluster Analysis,” Hoboken, New Jersey: John Wiley and Sons; 1990.
   Google Scholar
• T N Raymond, and J Han, “Efficient and effective clustering methods for spatial data mining,” in Proceedings of VLDB Conference, Santiago, Chile, pp. 144–55, 1994.
   Google Scholar
• J R Quinlan, “C4.5: Programs for Machine Learning,” San Mateo, CA: Morgan Kaufmann; 1993.
   Google Scholar
• S Russel, and P Norvig, “Artificial Intelligence: A Modern Approach,” Upper Saddle River, New Jersey: Prentice Hall; 1995.
   Google Scholar
• R Agrawal, T Imielinski, and A Swami, “Database mining: A performance perspective,” IEEE Transactions on Knowledge and Data Engineering, Vol. 5, pp. 914–25, 1993.
   Google Scholar
• S J Yen, and A L Chen, “An efficient algorithm for deriving compact rules from databases,” in Proceedings of the 4th International Conference on Database Systems for Advanced Applications, pp. 364–71, 1995.
   Google Scholar
• P M Murphy, and D W Aha, UCI Repository of Machine Learning Databases, (Machine-Readable Data Repository); Department of Information and Computer Science, University of California, Irvine, CA, USA, 1998. Available online: http://archive.ics.uci.edu/ml/. [Last cited on 2010 Jan 01].
   Google Scholar
• L Prechelt, “PROBEN1-A set of neural network benchmark problems and benchmarking rules,” Fakult at Inf., Univ. Karlsruhe, Karlsruhe, Germany, Technical Report, 1994.
   Google Scholar
• R Setiono, “Techniques for extracting rules from artificial neural networks,” Plenary lecture presented at the 5th International Conference on Soft Computing and Information Systems, lizuka, Japan, October 1998.
   Google Scholar
• L Breiman, J Friedman, R Olshen, and C Stone, “Classification and Regression Trees,” Wadsworth and Brooks, Monterey, CA, 1984.
   Google Scholar
• D T Pham, and M S Aksoy, “Rules: A simple rule extraction system,” Expert Systems with Applications, Vol. 8, pp. 59–65, 1995.
   Google Scholar
• D T Pham, and M S Aksoy, “An algorithm for automatic rule induction,” Artificial Intelligence in Engineering, Vol. 8, pp. 277–82, 1994.
   Google Scholar
• J Cendrowska, “PRISM: An algorithm for inducting modular rules,” International Journal of Man-Machine Studies, Vol. 27, pp. 349–70, 1987.
   Google Scholar