A Novel Approach of Ensemble Methods Using the Stacked Generalization for High-dimensional Datasets

REFERENCES

• S. Reid, “A review of heterogeneous ensemble methods,” Dep. Comput. Sci. Univ. Color. Boulder, pp. 1–10, 2007.
   Google Scholar
• K. Vinay, A. Rao, and G. Hemantha Kumar, “Computerized analysis of classification of lung nodules and comparison between homogeneous and heterogeneous ensemble of classifier model,” in Proceedings - 2011 3rd National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics, NCVPRIPG 2011, 2011, pp. 231–234.
   Google Scholar
• V. J. Kadam and S. M. Jadhav, Feature ensemble learning based on sparse autoencoders for diagnosis of Parkinson’s disease, vol. 810, no. January 2019. Springer Singapore, 2019.
   Google Scholar
• R. Tugay and Ş. G. Ögüdücü, “Demand prediction using machine learning methods and stacked generalization,” DATA 2017 - Proc. 6th Int. Conf. Data Sci. Technol. Appl., pp. 216–222, 2020.
   Google Scholar
• M. Massaoudi, S. S. Refaat, I. Chihi, M. Trabelsi, F. S. Oueslati, and H. Abu-Rub, “A novel stacked generalization ensemble-based hybrid LGBM-XGB-MLP model for short-term load forecasting,” Energy, vol. 214, p. 118874, 2021. DOI: 10.1016/j.energy.2020.118874.
   Web of Science ®Google Scholar
• S. Geman, E. Bienenstock, and R. Doursat, “Neural networks and the bias/variance dilemma,” Neural Comput., vol. 4, no. 1, pp. 1–58, 1992. DOI: 10.1162/neco.1992.4.1.1.
   Web of Science ®Google Scholar
• S. Sabra, K. M. Malik, M. Afzal, V. Sabeeh, and A. Charaf Eddine, “A hybrid knowledge and ensemble classification approach for prediction of venous thromboembolism,” Expert Syst., vol. 37, no. 1, pp. 1–14, 2019.
   Web of Science ®Google Scholar
• D. Wolpert, “Stacked generalization,” Neural Networks, Vol. 5, pp. 241–259, 1992. DOI: 10.1016/S0893-6080(05)80023-1.
   Web of Science ®Google Scholar
• L. Breiman, “Stacked regressions,” Mach. Learn., vol. 24, pp. 49–64, 1996.
   Web of Science ®Google Scholar
• N. Hatami and R. Ebrahimpour, “Combining multiple classifiers: diversify with boosting and combining by stacking,” IJCSNS Int. J. Comput. Sci. Netw. Secur., vol. 7, no. 1,pp. 127–131, 2007.
   Google Scholar
• M. J. Van der Laan, and S. Dudoit. “Unified cross-validation methodology for selection among estimators and a general cross-validated adaptive epsilon-net estimator: Finite sample oracle inequalities and examples,” 2003.
   Google Scholar
• M. J. Van der Laan, S. Dudoit, and A. W. van der Vaart, “The Cross-Validated Adaptive Epsilon-Net Estimator,” 2004.
   Google Scholar
• Y. Freund, and R. E. Schapire, “Experiments with a new boosting algorithm,” in In Proceedings of the Thirtheen International Conference on Machine Learning, pp. 148–156, 1996.
   Google Scholar
• M. P. Sesmero, A. I. Ledezma, and A. Sanchis, “Generating ensembles of heterogeneous classifiers using stacked generalization,” Wiley Interdiscip. Rev. Data Min. Knowl. Discov., vol. 5, no. 1, pp. 21–34, 2015. DOI: 10.1002/widm.1143.
   Web of Science ®Google Scholar
• A. Ozcift, “SVM feature selection based rotation forest ensemble classifiers to improve computer-aided diagnosis of Parkinson disease,” J. Med. Syst., vol. 36, no. 4,pp. 2141–2147, 2012. DOI: 10.1007/s10916-011-9678-1.
   PubMed Web of Science ®Google Scholar
• A. A. Akinyelu and A. O. Adewumi, “Classification of phishing email using random forest machine learning technique,” J. Appl. Math., vol. 2014, 2014. DOI: 10.1155/2014/425731.
   Google Scholar
• A. Onan, S. Korukoğlu, and H. Bulut, “A hybrid ensemble pruning approach based on consensus clustering and multi-objective evolutionary algorithm for sentiment classification,” Inf. Process. Manag., vol. 53, no. 4, pp. 814–833, 2017. DOI: 10.1016/j.ipm.2017.02.008.
   Web of Science ®Google Scholar
• K. Kumar, D. D. Shrimankar, and N. Singh, “Event BAGGING: A novel event summarization approach in multiview surveillance videos,” Proc. 2017 Int. Conf. Innov. Electron. Signal Process. Commun. IESC 2017, pp. 106–111, 2017.
   Google Scholar
• A. Onan, “An ensemble scheme based on language function analysis and feature engineering for text genre classification,” Journal of Information Science., vol. 44, no. 1,pp. 28–47, 2018. DOI: 10.1177/0165551516677911.
   Web of Science ®Google Scholar
• A. Onan and M. A. Tocoglu, “Satire identification in Turkish news articles based on ensemble of classifiers,” Turkish J. Electr. Eng. Comput. Sci., vol. 28, no. 2, pp. 1086–1106, 2020. DOI: 10.3906/elk-1907-11.
   Web of Science ®Google Scholar
• M. S. Al-Daweri, S. Abdullah, and K. A. Z. Ariffin, “A homogeneous ensemble based dynamic artificial neural network for solving the intrusion detection problem,” Int. J. Crit. Infrastruct. Prot., vol. 34, p. 100449, 2021. DOI: 10.1016/j.ijcip.2021.100449.
   Web of Science ®Google Scholar
• M. Y. Hu and C. Tsoukalas, “Explaining consumer choice through neural networks: The stacked generalization approach,” Eur. J. Oper. Res., vol. 146, no. 3, pp. 650–660, 2003. DOI: 10.1016/S0377-2217(02)00368-5.
   Web of Science ®Google Scholar
• S. Q. Wang, J. Yang, and K. C. Chou, “Using stacked generalization to predict membrane protein types based on pseudo-amino acid composition,” J. Theor. Biol., vol. 242, no. 4, pp. 941–946, 2006. DOI: 10.1016/j.jtbi.2006.05.006.
   PubMed Web of Science ®Google Scholar
• B. Qian and K. Rasheed, “Foreign exchange market prediction with multiple classifiers,” J. Forecast., vol. 29, no. 3,pp. 271–284, 2010.
   Web of Science ®Google Scholar
• L. Jonsson, D. Broman, K. Sandahl, and S. Eldh, “Towards automated anomaly report assignment in large complex systems using stacked generalization,” Proc. - IEEE 5th Int. Conf. Softw. Testing, Verif. Validation, ICST 2012,pp. 437–446, 2012.
   Google Scholar
• Y. Kim and E. Riloff, “Stacked generalization for medical concept extraction from clinical notes,” in Proceedings of the 2015 Workshop on Biomedical Natural Language Processing (BioNLP 2015), 2015, pp. 61–70.
   Google Scholar
• Z. Ma, P. Wang, Z. Gao, R. Wang, and K. Khalighi, “Ensemble of machine learning algorithms using the stacked generalization approach to estimate the warfarin dose,” PLoS One, Vol. 13, no. 10, pp. 1–12, 2018.
   Web of Science ®Google Scholar
• B. Wang, S. Fan, P. Jiang, T. Xing, Z. Fang, and Q. Wen, “Research on predicting the productivity of cutter suction dredgers based on data mining with model stacked generalization,” Ocean Eng., vol. 217, p. 108001, 2020. DOI: 10.1016/j.oceaneng.2020.108001.
   Web of Science ®Google Scholar
• A. Onan, and M. A. Tocoglu, “A term weighted neural language model and stacked bidirectional LSTM based framework for sarcasm identification,” IEEE Access, Vol. 9,pp. 7701–7722, 2021. DOI: 10.1109/ACCESS.2021.304-9734.
   Web of Science ®Google Scholar
• A. I. Naimi and L. B. Balzer, “Stacked generalization: An introduction to super learning,” Eur. J. Epidemiol., vol. 33, no. 5, pp. 459–464, 2018. DOI: 10.1007/s10654-018-0390-z.
   PubMed Web of Science ®Google Scholar
• I. H. Witten and E. Frank, “Data mining: practical machine learning tools and techniques with Java implementations,” Acm Sigmod Record, Vol. 31, no. 1, pp. 76–77, 2002. DOI: 10.1145/507338.507355.
   Google Scholar
• S. Wang, W. Pedrycz, Q. Zhu, and W. Zhu, “Unsupervised feature selection via maximum projection and minimum redundancy,” Knowledge-Based Syst., vol. 75, pp. 19–29, 2015. DOI: 10.1016/j.knosys.2014.11.008.
   Web of Science ®Google Scholar
• H. A. Le Thi, V. V. Nguyen, and S. Ouchani, “Gene selection for cancer classification using DCA,” Lect. Notes Comput. Sci. (Including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 5139, LNAI, pp. 62–72, 2008.
   Google Scholar
• G. Brown, “A new perspective for information theoretic feature selection,” J. Mach. Learn. Res., vol. 5, no. 1,pp. 49–56, 2009.
   Google Scholar
• W. C. Lin, Y. H. Lu, and C. F. Tsai, “Feature selection in single and ensemble learning-based bankruptcy prediction models,” Expert Syst., vol. 36, no. 1, pp. 1–8, 2019.
   Web of Science ®Google Scholar
• J. Vanschoren, J. N. Van Rijn, B. Bischl, and L. Torgo, “OpenML: Networked science in machine learning,” ACM SIGKDD Explor. Newsl., vol. 15, no. 2, pp. 49–60, 2014. DOI: 10.1145/2641190.2641198.
   Google Scholar
• G. Casalicchio, et al., “OpenML: An R package to connect to the machine learning platform OpenML,” Comput. Stat., vol. 34, no. 3, pp. 977–991, 2019. DOI: 10.1007/s00180-017-0742-2.
   Web of Science ®Google Scholar
• Z. Zhao, F. Morstatter, S. Sharma, S. Alelyani, A. Anand, and H. Liu, “Advancing feature selection research,” ASU Featur. Sel. Repos., pp. 1–28, 2010.
   Google Scholar
• J. Ramey, “Datamicroarray: Collection of data sets for classification,” Datamicroarray: collection of data sets for classification, Vol. 2016 (2016).
   Google Scholar
• F. Wilcoxon, “Individual comparisons by ranking methods,” Biometrics Bull., vol. 1, no. 6, p. 80, 1945. DOI: 10.2307/3001968.
   Web of Science ®Google Scholar
• A. Ozcift and A. Gulten, “Classifier ensemble construction with rotation forest to improve medical diagnosis performance of machine learning algorithms,” Comput. Methods Programs Biomed., vol. 104, no. 3, pp. 443–451, 2011. DOI: 10.1016/j.cmpb.2011.03.018.
   PubMed Web of Science ®Google Scholar
• Ioannis Psorakis, Theodoros Damoulas, and Mark A Girolami, “Multiclass relevance vector machines: Sparsity and accuracy,” IEEE Transactions on Neural Networks, vol. 21, no. 10, pp. 1588–1598, 2010. DOI: 10.1109/TNN.2010.2064787.
   PubMedGoogle Scholar
• A. Caliskan, H. Badem, A. Baştürk, and M. E. Yüksel, “Diagnosis of the Parkinson disease by using deep neural network classifier,” Istanbul Univ. - J. Electr. Electron. Eng., vol. 17, no. 2, pp. 3311–3318, 2017.
   Google Scholar
• Hyunjin Kwon, Jinhyeok Park, and Youngho Lee, “Stacking ensemble technique for classifying breast cancer,” Healthcare Informatics Research, vol. 25, no. 4, pp. 283, 2019. DOI: 10.4258/hir.2019.25.4.283.
   PubMed Web of Science ®Google Scholar
• Vikas Chaurasia, and Saurabh Pal, “Applications of machine learning techniques to predict diagnostic breast cancer,” SN Computer Science, vol. 1, no. 5, pp. 1–11, 2020. DOI: 10.1007/s42979-020-00296-8.
   Google Scholar
• Satish Kumar Kalagotla, Suryakanth V Gangashetty, and Kanuri Giridhar, “A novel stacking technique for prediction of diabetes,” Computers in Biology and Medicine, vol. 135, pp. 104554, 2021. DOI: 10.1016/j.compbiomed.2021.104554.
   PubMed Web of Science ®Google Scholar
• Seyedamin Pouriyeh, Sara Vahid, Giovanna Sannino, Giuseppe De Pietro, Hamid Arabnia, and Juan Gutierrez, “A comprehensive investigation and comparison of Machine Learning Techniques in the domain of heart disease,” in 2017 IEEE Symposium on Computers and Communications (ISCC), Heraklion, Greece , 2017, pp. 204–207.
   Google Scholar