Analysis of generalizability on predicting COVID-19 from chest X-ray images using pre-trained deep models

ABSTRACT

Machine learning methods have been extensively employed to predict COVID-19 using chest X-ray images in numerous studies. However, a machine learning model must exhibit robustness and provide reliable predictions for diverse populations, beyond those used in its training data, to be truly valuable. Unfortunately, the assessment of model generalisability is frequently overlooked in current literature. In this study, we investigate the generalisability of three classification models – ResNet50v2, MobileNetv2, and Swin Transformer – for predicting COVID-19 using chest X-ray images. We adopt three concurrent approaches for evaluation: the internal-and-external validation procedure, lung region cropping, and image enhancement. The results show that the combined approaches allow deep models to achieve similar internal and external generalisation capability.

KEYWORDS:

• COVID-19
• X-ray
• machine learning

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. https://github.com/dirtmaxim/lungs-finder

2. https://keras.io/examples/vision/swin_transformers/

3. https://www.kaggle.com/c/rsna-pneumonia-detection-challenge

4. https://github.com/agchung/Actualmed-COVID-chestxray-dataset

5. Figure 1-COVID-chestxray-datasethttps://github.com/agchung/Figure 1-COVID-chestxray-dataset

Additional information

Funding

The present work is the result of the Research and Development (R&D) project 001/2020, signed with Federal University of Amazonas and FAEPI, Brazil, which has funding from Samsung, using resources from the Informatics Law for the Western Amazon (Federal Law no 8.387/1991), and its disclosure is in accordance with article 39 of Decree No. 10.521/2020.

Notes on contributors

Natalia de Sousa Freire

Natalia de Sousa Freire is currently a Software Engineering student at the Federal University of Amazonas (UFAM). His main research interests include the areas of machine learning and computer vision.

Pedro Paulo de Souza Leo

Pedro Paulo de Souza Leão obtained his Bachelor's degree in Software Engineering from the Federal University of Amazonas (Brazil) in 2023. His main research interest is machine learning.

Leonardo Albuquerque Tiago

Leonardo de Albuquerque Tiago is currently pursuing a Bachelor's degree in Software Engineering at Federal University of Amazonas (Brazil). His main research interests are machine learning and software testing.

Alberto de Almeida Campos Gonalves

Alberto de Almeida Campos Gonçalves received his B.S. degree in Computer Science from the Federal University of Amazonas in 2022. His research interests include the areas of machine learning and computer vision.

Rafael Albuquerque Pinto

Rafael Albuquerque Pinto received his B.S. degree in Computer Science from the Federal University of Roraima (UFRR) in 2017 and his M.Sc. degree in Informatics from the Federal University of Amazonas (UFAM) in 2022. He is currently pursuing a Ph.D. degree in Informatics at UFAM, focusing his research on biosignals using machine learning techniques.

Eulanda Miranda dos Santos

Eulanda Miranda dos Santos is an Associate Professor in the Institute of Computing (IComp) of the Federal University of Amazonas. She received a B.Sc. degree in Informatics from Federal University of Para (Brazil), a M.Sc. degree in Informatics from Federal University of Paraiba (Brazil) and a Ph.D. degree in Engineering from École de Technologie Supérieure, University of Quebec (Canada) in 1999, 2002 and 2008, respectively. Her research interests include pattern recognition, machine learning and computer vision.

Eduardo Souto

Eduardo Souto received the Ph.D. degree in computer science from the Federal University of Pernambuco (UFPE), Recife, Brazil, in 2007. He is currently an Associate Professor with the Institute of Computing, Federal University of Amazonas (UFAM). He is also the head of the Emerging Technologies and System Security (ETSS) Research Group. His research interests include the areas of applied machine learning, internet of things, and network security.