597
Views
0
CrossRef citations to date
0
Altmetric
Research Article

TfrAdmCov: a robust transformer encoder based model with Adam optimizer algorithm for COVID-19 mutation prediction

ORCID Icon & ORCID Icon
Article: 2365334 | Received 06 Feb 2024, Accepted 03 Jun 2024, Published online: 12 Jun 2024

References

  • Abbas, M. E., Chengzhang, Z., Fathalla, A., & Xiao, Y. (2022). End-to-end antigenic variant generation for H1N1 influenza HA protein using sequence to sequence models. PLoS ONE, 17(3 March), 1–14. https://doi.org/10.1371/journal.pone.0266198
  • Ahmad, G. N., Fatima, H., Ullah, S., Saidi, A. S., & Imdadullah. (2022). Efficient medical diagnosis of human heart diseases using machine learning techniques with and without GridSearchCV. IEEE Access, 10(March), 80151–80173. https://doi.org/10.1109/ACCESS.2022.3165792
  • Ahmed, I., & Jeon, G. (2022). Enabling artificial intelligence for genome sequence analysis of COVID-19 and alike viruses. Interdisciplinary Sciences: Computational Life Sciences, 14(2), 504–519. https://doi.org/10.1007/s12539-021-00465-0
  • Anonymous. (2023a). clustalw. https://www.genome.jp/tools-bin/clustalw.
  • Anonymous. (2023b). COVID-19 S protein dataset. https://www.ncbi.nlm.nih.gov/datasets/taxonomy/2697049/.
  • Asgari, E., & Mofrad, M. R. K. (2015). Continuous distributed representation of biological sequences for deep proteomics and genomics. PLoS ONE, 10(11), e0141287. https://doi.org/10.1371/journal.pone.0141287
  • Barnes, C. O., West, A. P., Huey-Tubman, K. E., Hoffmann, M. A. G., Sharaf, N. G., Hoffman, P. R., Koranda, N., Gristick, H. B., Gaebler, C., Muecksch, F., Lorenzi, J. C. C., Finkin, S., Hägglöf, T., Hurley, A., Millard, K. G., Weisblum, Y., Schmidt, F., Hatziioannou, T., Bieniasz, P. D., … Bjorkman, P. J. (2020). Structures of human antibodies bound to SARS-CoV-2 spike reveal common epitopes and recurrent features of antibodies. Cell, 182(4), 828–842.e16. https://doi.org/10.1016/j.cell.2020.06.025
  • Cai, C., Li, J., Xia, Y., & Li, W. (2024). FluPMT: Prediction of predominant strains of influenza a viruses via multi-task learning. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 1–11. https://doi.org/10.1109/TCBB.2024.3378468
  • Chakraborty, G. S., Singh, D., Rakhra, M., Batra, S., & Singh, A. (2022). Covid-19 and diabetes risk prediction for diabetic patient using advance machine learning techniques and fuzzy inference system. Proceedings of 5th International Conference on Contemporary Computing and Informatics, 14-16 December, IC3I 2022, 2022, 1212–1219. https://doi.org/10.1109/IC3I56241.2022.10073256
  • Chicco, D., & Jurman, G. (2020). The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics, 21(1), 1–13. https://doi.org/10.1186/s12864-019-6413-7
  • Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. ArXiv:1412.3555, 1–9. http://arxiv.org/abs/1412.3555.
  • Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.
  • Cui, J., Li, F., & Shi, Z.-L. (2019). Origin and evolution of pathogenic coronaviruses. Nature Reviews Microbiology, 17(3), 181–192. https://doi.org/10.1038/s41579-018-0118-9
  • de Wit, J. J., & Cook, J. K. A. (2020). Spotlight on avian coronaviruses. Avian Pathology, 49(4), 313–316. https://doi.org/10.1080/03079457.2020.1761010
  • ElAraby, M. E., Elzeki, O. M., Shams, M. Y., Mahmoud, A., & Salem, H. (2022). A novel gray-scale spatial exploitation learning Net for COVID-19 by crawling internet resources. Biomedical Signal Processing and Control, 73(2022), 103441. https://doi.org/10.1016/j.bspc.2021.103441
  • Elzeki, O. M., Elfattah, M. A., Salem, H., Hassanien, A. E., & Shams, M. (2021). A novel perceptual two layer image fusion using deep learning for imbalanced COVID-19 dataset. PeerJ Computer Science, 7:e364, 1–35. https://doi.org/10.7717/PEERJ-CS.364
  • Elzeki, O. M., Shams, M., Sarhan, S., Elfattah, M. A., & Hassanien, A. E. (2021). COVID-19: A new deep learning computer-aided model for classification. PeerJ Computer Science, 7:e358, 1–33. https://doi.org/10.7717/peerj-cs.358
  • Feng, J., Xu, H., Mannor, S., & Yan, S. (2014). Robust logistic regression and classification. Advances in Neural Information Processing Systems, 27, 253–261.
  • Gage, A., Brunson, K., Morris, K., Wallen, S. L., Dhau, J., Gohel, H., & Kaushik, A. (2021). Perspectives of manipulative and high-performance nanosystems to manage consequences of emerging new severe acute respiratory syndrome coronavirus 2 variants. Frontiers in Nanotechnology, 3(June), 1–7. https://doi.org/10.3389/fnano.2021.700888
  • Galassi, A., Lippi, M., & Torroni, P. (2021). Attention in natural language processing. IEEE Transactions on Neural Networks and Learning Systems, 32(10), 4291–4308. https://doi.org/10.1109/TNNLS.2020.3019893
  • Guo, G., Wang, H., Bell, D., Bi, Y., & Greer, K. (2003). KNN Model-based approach in classification. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2888, 986–996. https://doi.org/10.1007/978-3-540-39964-3_62
  • Hai-Dong, L., Ya-Juan, Y., & Lu, L. (2022). In the context of COVID-19: the impact of employees' risk perception on work engagement. Connection Science, 34(1), 1367–1383. https://doi.org/10.1080/09540091.2022.2071839
  • Haimed, A. M. A., Saba, T., Albasha, A., Rehman, A., & Kolivand, M. (2021). Viral reverse engineering using artificial intelligence and big data COVID-19 infection with long short-term memory (LSTM). Environmental Technology & Innovation, 22, 101531. https://doi.org/10.1016/j.eti.2021.101531
  • Hassan, E., Shams, M. Y., Hikal, N. A., & Elmougy, S. (2023). COVID-19 diagnosis-based deep learning approaches for COVIDx dataset: A preliminary survey. In G. K. Mostefaoui, S. M. R. Islam, & F. Tariq (Eds.), Artificial intelligence for disease diagnosis and prognosis in smart healthcare (pp.107–122)
  • Hassan, E., Shams, M. Y., Hikal, N. A., & Elmougy, S. (2024). Detecting COVID-19 in chest CT images based on several pre-trained models. Multimedia Tools and Applications, 1–21. https://doi.org/10.1007/s11042-023-17990-3
  • Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
  • Hossain, M. K., Hassanzadeganroudsari, M., & Apostolopoulos, V. (2021b). The emergence of new strains of SARS-CoV-2. What does it mean for COVID-19 vaccines? Expert Review of Vaccines, 20(6), 635–638. https://doi.org/10.1080/14760584.2021.1915140
  • Hossain, M. S., Pathan, A. Q. M. S. U., Islam, M. N., Tonmoy, M. I. Q., Rakib, M. I., Munim, M. A., Saha, O., Fariha, A., Al Reza, H., Roy, M., Bahadur, N. M., & Rahaman, M. M. (2021a). Genome-wide identification and prediction of SARS-CoV-2 mutations show an abundance of variants: Integrated study of bioinformatics and deep neural learning. Informatics in Medicine Unlocked, 27, 100798. https://doi.org/10.1016/j.imu.2021.100798
  • Huang, Y., Yang, C., Xu, X., Xu, W., & Liu, S. (2020). Structural and functional properties of SARS-CoV-2 spike protein: Potential antivirus drug development for COVID-19. Acta Pharmacologica Sinica, 41(9), 1141–1149. https://doi.org/10.1038/s41401-020-0485-4
  • Jackson, C. B., Farzan, M., Chen, B., & Choe, H. (2022). Mechanisms of SARS-CoV-2 entry into cells. Nature Reviews Molecular Cell Biology, 23(1), 3–20. https://doi.org/10.1038/s41580-021-00418-x
  • Jaimes, J. A., André, N. M., Chappie, J. S., Millet, J. K., & Whittaker, G. R. (2020). Phylogenetic analysis and structural modeling of SARS-CoV-2 spike protein reveals an evolutionary distinct and proteolytically sensitive activation loop. Journal of Molecular Biology, 432(10), 3309–3325. https://doi.org/10.1016/j.jmb.2020.04.009
  • Kalyan, K. S., Rajasekharan, A., & Sangeetha, S. (2021). AMMUS: A Survey of Transformer-based Pretrained Models in Natural Language Processing. ArXiv Preprint ArXiv:2108.05542., 1–42. http://arxiv.org/abs/2108.05542.
  • Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, IJCAI’95, Morgan Kaufmann, United States, Montreal, QC, Canada, 20–25 August, pp. 1137–1143.
  • Li, M., Zhao, B., Yin, R., Lu, C., Guo, F., & Zeng, M. (2023). Graphlncloc: Long non-coding RNA subcellular localization prediction using graph convolutional networks based on sequence to graph transformation. Briefings in Bioinformatics, 24(1), 1–12. https://doi.org/10.1093/bib/bbac565
  • Lopez-Rincon, A., Perez-Romero, C. A., Tonda, A., Mendoza-Maldonado, L., Claassen, E., Garssen, J., & Kraneveld, A. D. (2021). Design of specific primer sets for the detection of B.1.1.7, B.1.351 and P.1 SARS-CoV-2 variants using deep learning. BioRxiv, 70, 2021.01.20.427043. https://doi.org/10.1101/2021.01.20.427043
  • Luque, A., Carrasco, A., Martín, A., & de las Heras, A. (2019). The impact of class imbalance in classification performance metrics based on the binary confusion matrix. Pattern Recognition, 91, 216–231. https://doi.org/10.1016/j.patcog.2019.02.023
  • Madhi, S. A., Kwatra, G., Myers, J. E., Jassat, W., Dhar, N., Mukendi, C. K., Nana, A. J., Blumberg, L., Welch, R., Ngorima-Mabhena, N., & Mutevedzi, P. C. (2022). Population immunity and Covid-19 severity with Omicron Variant in South Africa. New England Journal of Medicine, 386(14), 1314–1326. https://doi.org/10.1056/NEJMoa2119658
  • Mbow, M., Koide, H., & Sakurai, K. (2021). An intrusion detection system for imbalanced dataset based on deep learning. 2021 Ninth International Symposium on Computing and Networking (CANDAR), 23-26 November 2021, 38–47. https://doi.org/10.1109/CANDAR53791.2021.00013
  • Memon, N., Patel, S. B., & Patel, D. P. (2019). Comparative analysis of artificial neural network and XGBoost algorithm for PolSAR image classification. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11941(LNCS(Lc)), 452–460. https://doi.org/10.1007/978-3-030-34869-4_49
  • Mohamed, T., Sayed, S., Salah, A., & Houssein, E. H. (2021). Long short-term memory neural networks for RNA viruses mutations prediction. Mathematical Problems in Engineering, 2021. https://doi.org/10.1155/2021/9980347
  • Nawaz, M. S., Fournier-Viger, P., Shojaee, A., & Fujita, H. (2021). Using artificial intelligence techniques for COVID-19 genome analysis. Applied Intelligence, 51(5), 3086–3103. https://doi.org/10.1007/s10489-021-02193-w
  • Norouzi, M., Fleet, D. J., & Salakhutdinov, R. (2012). Hamming distance metric learning. Advances in Neural Information Processing Systems, 2, 1061–1069.
  • Pacal, I. (2024a). A novel Swin transformer approach utilizing residual multi-layer perceptron for diagnosing brain tumors in MRI images. International Journal of Machine Learning and Cybernetics, 1–21. https://doi.org/10.1007/s13042-024-02110-w
  • Pacal, I. (2024b). Maxcervixt: A novel lightweight vision transformer-based approach for precise cervical cancer detection. Knowledge-Based Systems, 289(October 2023). https://doi.org/10.1016/j.knosys.2024.111482
  • Paszke, A., Gross, S., Chintala, S., Chanan, G., & Yang, E. (2017). Automatic differentiation in pytorch. 31st conference on Neural Information Processing Systems (NIPS 2017), Nips, 1–4. https://openreview.net/forum?id=BJJsrmfCZ .
  • Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, D. (2011). Scikit-learn: Machine learning in Python Fabian. Journal of Machine Learning Research, 12, 2825–2830.
  • Qin, L., Ding, X., Li, Y., Chen, Q., Meng, J., & Jiang, T. (2021). Co-mutation modules capture the evolution and transmission patterns of SARS-CoV-2. Briefings in Bioinformatics, 00(May), 1–10. https://doi.org/10.1093/bib/bbab222
  • Raheja, S., Kasturia, S., Cheng, X., & Kumar, M. (2023). Machine learning-based diffusion model for prediction of coronavirus-19 outbreak. Neural Computing and Applications, 35(19), 13755–13774. https://doi.org/10.1007/s00521-021-06376-x
  • Rashid, Z.Z., Othman, S. N., Abdul Samat, M. N., Ali, U. K., & Ken, W. K. (2020). Diagnostic performance of COVID-19 serology assays. Malaysian Journal of Pathology, 42(1), 13–21.
  • Saha, I., Ghosh, N., Maity, D., Sharma, N., Sarkar, J. P., & Mitra, K. (2020). Genome-wide analysis of Indian SARS-CoV-2 genomes for the identification of genetic mutation and SNP. Infection, Genetics and Evolution, 85, 104457. https://doi.org/10.1016/j.meegid.2020.104457
  • Salama, M. A., Hassanien, A. E., & Mostafa, A. (2016). The prediction of virus mutation using neural networks and rough set techniques. Eurasip Journal on Bioinformatics and Systems Biology, 2016(1), 1–11. https://doi.org/10.1186/s13637-016-0042-0
  • Sasirekha, K., & Baby, P. (2013). Agglomerative hierarchical clustering algorithm – A review. International Journal of Scientific and Research Publications, 3(3), 83–85.
  • Serena Low, W. C., Chuah, J. H., Tee, C. A. T. H., Anis, S., Shoaib, M. A., Faisal, A., Khalil, A., & Lai, K. W. (2021). An overview of deep learning techniques on chest X-Ray and CT scan identification of COVID-19. Computational and Mathematical Methods in Medicine, 2021. https://doi.org/10.1155/2021/5528144
  • Shaikh, F., Andersen, M. B., Sohail, M. R., Mulero, F., Awan, O., Dupont-Roettger, D., Kubassova, O., Dehmeshki, J., & Bisdas, S. (2021). Current landscape of imaging and the potential role for artificial intelligence in the management of COVID-19. Current Problems in Diagnostic Radiology, 50(3), 430–435. https://doi.org/10.1067/j.cpradiol.2020.06.009
  • Sharma, A., Ahmad Farouk, I., & Lal, S. K. (2021). Covid-19: A review on the novel coronavirus disease evolution, transmission, detection, control and prevention. Viruses, 13(2), 1–25. https://doi.org/10.3390/v13020202
  • Shereen, M. A., Khan, S., Kazmi, A., Bashir, N., & Siddique, R. (2020). COVID-19 infection: Emergence, transmission, and characteristics of human coronaviruses. Journal of Advanced Research, 24, 91–98. https://doi.org/10.1016/j.jare.2020.03.005
  • Shiehzadegan, S., Alaghemand, N., Fox, M., & Venketaraman, V. (2021). Analysis of the Delta Variant B.1.617.2 COVID-19. Clinics and Practice, 11(4), 778–784. https://doi.org/10.3390/clinpract11040093
  • Shrestha, H., Dhasarathan, C., Kumar, M., Nidhya, R., Shankar, A., & Kumar, M. (2022). A deep learning based convolution neural network-DCNN approach to detect brain tumor (pp. 115–127). https://doi.org/10.1007/978-981-16-6887-6_11
  • Sohrabi, C., Alsafi, Z., O’Neill, N., Khan, M., Kerwan, A., Al-Jabir, A., Iosifidis, C., & Agha, R. (2020). World Health Organization declares global emergency: A review of the 2019 novel coronavirus (COVID-19). International Journal of Surgery, 76, 71–76. https://doi.org/10.1016/j.ijsu.2020.02.034
  • Sokhansanj, B. A., & Rosen, G. L. (2022). Predicting COVID-19 disease severity from SARS-CoV-2 spike protein sequence by mixed effects machine learning. Computers in Biology and Medicine, 149. https://doi.org/10.1016/j.compbiomed.2022.105969
  • Suri, J. S., Puvvula, A., Biswas, M., Majhail, M., Saba, L., Faa, G., Singh, I. M., Oberleitner, R., Turk, M., Chadha, P. S., Johri, A. M., Sanches, J. M., Khanna, N. N., Viskovic, K., Mavrogeni, S., Laird, J. R., Pareek, G., Miner, M., Sobel, D. W., … Naidu, S. (2020). COVID-19 pathways for brain and heart injury in comorbidity patients: A role of medical imaging and artificial intelligence-based COVID severity classification: A review. Computers in Biology and Medicine, 124(January), 103960. https://doi.org/10.1016/j.compbiomed.2020.103960
  • Tang, L., Wu, T., Chen, X., Wen, S., Zhou, W., Zhu, X., & Xiang, Y. (2024). How COVID-19 impacts telehealth: An empirical study of telehealth services, users and the use of metaverse. Connection Science, 36(1). https://doi.org/10.1080/09540091.2023.2282942
  • Tarek, Z., Shams, M. Y., Towfek, S. K., Alkahtani, H. K., Ibrahim, A., Abdelhamid, A. A., Eid, M. M., Khodadadi, N., Abualigah, L., Khafaga, D. S., & Elshewey, A. M. (2023). An optimized model based on deep learning and gated recurrent unit for COVID-19 death prediction. Biomimetics, 8(7), 552. https://doi.org/10.3390/biomimetics8070552
  • Thara, D. K., Prema Sudha, B. G., & Xiong, F. (2019). Auto-detection of epileptic seizure events using deep neural network with different feature scaling techniques. Pattern Recognition Letters, 128, 544–550. https://doi.org/10.1016/j.patrec.2019.10.029
  • Thölke, P., Jose, Y., Ramos, M., & Abdelhedi, H. (2022). Class imbalance should not throw you off balance: choosing the right classifiers and performance metrics for brain decoding with imbalanced data. BioRxiv, 1–35. https://doi.org/10.1101/2022.07.18.500262
  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł, & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 2017-Decem(Nips), 5999–6009.
  • Voita, E., Talbot, D., Moiseev, F., Sennrich, R., & Titov, I. (2020). Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. ACL 2019 – 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, 28 July – 2 August, 5797–5808. https://doi.org/10.18653/v1/p19-1580
  • Wang, R., Hozumi, Y., Yin, C., & Wei, G.-W. (2020). Mutations on COVID-19 diagnostic targets. Genomics, 112(6), 5204–5213. https://doi.org/10.1016/j.ygeno.2020.09.028
  • World Health Organization. (2023). Novel-Coronavirus-2019 Www.Who.Int. https://www.who.int/es/emergencies/diseases/novel-coronavirus-2019.
  • Wu, C., Yin, W., Jiang, Y., & Xu, H. E. (2022). Structure genomics of SARS-CoV-2 and its omicron variant: Drug design templates for COVID-19. Acta Pharmacologica Sinica, December 2021, 1–13. https://doi.org/10.1038/s41401-021-00851-w
  • Wu, F., Zhao, S., Yu, B., Chen, Y.-M., Wang, W., Song, Z.-G., Hu, Y., Tao, Z.-W., Tian, J.-H., Pei, Y.-Y., Yuan, M.-L., Zhang, Y.-L., Dai, F.-H., Liu, Y., Wang, Q.-M., Zheng, J.-J., Xu, L., Holmes, E. C., & Zhang, Y.-Z. (2020). A new coronavirus associated with human respiratory disease in China. Nature, 579(7798), 265–269. https://doi.org/10.1038/s41586-020-2008-3
  • Yin, R., Luo, Z., Zhuang, P., Zeng, M., Li, M., Lin, Z., & Kwoh, C. K. (2023). Vipal: A framework for virulence prediction of influenza viruses with prior viral knowledge using genomic sequences. Journal of Biomedical Informatics, 142(April). https://doi.org/10.1016/j.jbi.2023.104388
  • Yin, R., Luusua, E., Dabrowski, J., Zhang, Y., & Kwoh, C. K. (2020). Tempel: Time-series mutation prediction of influenza A viruses via attention-based recurrent neural networks. Bioinformatics (oxford, England), 36(9), 2697–2704. https://doi.org/10.1093/bioinformatics/btaa050
  • Yin, R., Thwin, N. N., Zhuang, P., Lin, Z., & Kwoh, C. K. (2022). IAV-CNN: A 2D convolutional neural network model to predict antigenic variants of influenza A virus. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 19(6), 3497–3506. https://doi.org/10.1109/TCBB.2021.3108971
  • Zaremba, W., Sutskever, I., & Vinyals, O. (2014). Recurrent Neural Network Regularization. ArXiv:1409.2329, 2013, 1–8. http://arxiv.org/abs/1409.2329.
  • Zhang, J., Xiao, T., Cai, Y., & Chen, B. (2021). Structure of SARS-CoV-2 spike protein. Current Opinion in Virology, 50, 173–182. https://doi.org/10.1016/j.coviro.2021.08.010
  • Zhou, B., Zhou, H., Zhang, X., Xu, X., Chai, Y., Zheng, Z., Kot, A. C., & Zhou, Z. (2023). TEMPO: A transformer-based mutation prediction framework for SARS-CoV-2 evolution. Computers in Biology and Medicine, 152, 106264. https://doi.org/10.1016/j.compbiomed.2022.106264