QSLRS-CNN: Qur'anic sign language recognition system based on convolutional neural networks

ABSTRACT

Deaf and dumb Muslims face educational barriers. They can't read, recite, or comprehend the Holy Qur'an, hence they can't practise Islamic ceremonies. This study proposes a CNN-based Qur'anic sign language recognition methodology. First, photos are used to train for dynamic and static gesture recognition. Second, preparing images diversifies datasets. Finally, CNN-based deep learning models extract and classify features. To teach the deaf and dumb Islamic ceremonies, the programme recognises Arabic sign language hand motions referring to dashed Qur'anic letters. Only 24,137 photos of the Holy Qur'an's 14 dashed letters were used in the trials from ArSL2018, a huge Arabic sign language collection. SMOTE raises training and testing accuracy to 98.31% and 97.67%, respectively, whereas the proposed model reaches 98.05% and 97.13%. RMU obtains 98.66% and 97.52% training and testing accuracy, whereas RMO achieves 98.37% and 97.36%.

KEYWORDS:

• Holy Qur'an
• Qur'anic sign language
• RMO
• convolutional neural network
• SMOTE
• sign language
• deep Learning
• feature extraction

Acknowledgment

The authors extend their appreciation to the Deanship of Scientific Research at King Khalid University.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The authors extend their appreciation to the Deanship of Scientific Research at King Khalid University for funding this work through large group Research Project under grant number [RGP2/246/44].

Notes on contributors

Hany A. AbdElghfar

Hany A. AbdElghfar received his B.Sc. and M.Sc. degrees from Systems and Computers Engineering, Faculty of Engineering, Al- Azhar University in Cairo, Egypt. His research interests include, Artificial intelligent, pattern recognition, machine learning, Deep Learning, E-Learning, Intelligence Systems and Computer Vision IOT systems. He is having 15 years of teaching and research experience at various reputed Universities of Egypt.

Abdelmoty M. Ahmed

Abdelmoty M. Ahmed received his B.Sc., M.Sc. and PhD degrees. His research interests include Digital image processing, Artificial intelligent, pattern recognition, Human Computer Interaction, Computer Graphics, machine learning, Deep Learning, E-Learning, Intelligence Systems Engineering, Computer Vision and IOT systems, he is senior lecturer in computer engineering department at College of Computer Science, King Khalid University, Abha, Saudi Arabia, he is also interested in researching the technical fields that serve deaf and dumb and also works in the automatic translation of the Arabic Sign Language. He is having 20 years of teaching and research experience at various reputed Universities of Egypt and Saudi Arabia. He has published more research articles in reputed SCI and scopus indexed journals and conferences.

Ali A. Alani

Ali A. Alani received his B.Sc. Degree in computer sciences from Diyala University, Diyala, Iraq in 2006 and M.Sc. Degree in Information Technology from Universiti Tenaga Nasional, Selangor, Malaysia in 2014. Recently, he is working as Assistant Lecturer in Department of computer sciences in university of Diyala, Diyala, Iraq. His research interests include Big data, Machine learning, Deep Learning and Computer vision.

Hammam M. AbdElaal

Hammam M. AbdElaal received his B.Sc. and M.Sc. degrees in computers & systems engineering from faculty of engineering, Al-Azhar University, Cairo, in 2005 and 2016, respectively, and the Ph.D. degree in computer engineering from the Computer Engineering Department, Faculty of Engineering, Minia University, Egypt in 2020. He has been a Doctor (Lecturer) at faculty of computer and information, Luxor University. His main areas of research interest are Machine learning Techniques, supervised Learning algorithms, Natural language processing, and Data Mining.

Belgacem Bouallegue

Belgacem Bouallegue received his B.Sc. and M.Sc. degrees from University of Monastir, Tunisia, and Ph.D. from Graduate School of Engineering Science and Technology, University of Southern Brittany in in Lorient, France with the cooperation of University of Monastir, Tunisia. He is currently an Assistant Professor in Department of Computer Engineering at College of Computer Science, King Khalid University, Saudi Arabia. His research interests include Integrated System Design, Fault Tolerance, HW/SW Co-design, Parallel Computers, Embedded Systems and IoT, Network on Chip NoC, AI, IPs and MPSoCs, Machine Learning, Deep Learning, Wireless Sensor Networks Security, and Cryptography. He is working in collaboration with Lab-STICC Laboratory, Lorient, France and LIP6, Computer Science Research Laboratory, PARIS Cedex 05, France.

Mahmoud M. Khattab

Mahmoud M. Khattab received his B.Sc. (2005) and M.Sc. (2009) degrees in computer science from faculty of computers and information, Menofiya University, Egypt. He earned his Ph.D. (2022) degree in computer science from kulliyyah (faculty) of information and communication technology, International Islamic University Malaysia (IIUM), Kuala Lumpur, Malaysia. The area of his research interest lies in super-resolution, image processing, pattern recognition, artificial intelligent, and computer vision. He is a lecturer in computer science department at King Khalid University (KKU), Saudi Arabia.

Hassan A. Youness

Hassan A. Youness received the B.Sc. and M.Sc. degrees from Assiut University, Assiut, Egypt, and the Ph.D. degree from the Graduate School of Information Science and Technology, Osaka University, Japan, with the cooperation of Ain Shams University, Egypt. He worked for IBM Company and Mentor Graphics, Egypt. He is currently Professor with Minia University, and also the Chairman of the Computers and Systems Engineering Department. His research interests include integrated system design, fault tolerance, HW/SW co-design, parallel computers, embedded systems, GPGPU, APU and MPSoCs, and homogeneous/heterogeneous systems.