VGG16-Based Feature Extraction for Arabic Alphabet Sign Language Classification to Support Qur'anic Tadarus Accessibility
DOI:
https://doi.org/10.52436/1.jutif.2025.6.4.4953Keywords:
Arabic Sign Language, Deep Learning, Qur'anic Accessibility, Transfer Learning, VGG16Abstract
This study addresses the limited availability of automated recognition systems for Arabic Alphabet Sign Language (ArSL), particularly in facilitating Qur’anic Tadarus for the deaf and hard-of-hearing community. While research on American and Indonesian sign languages has advanced significantly, ArSL studies, especially for static alphabet gestures, remain underrepresented. The aim of this research is to develop an accurate and efficient ArSL classifier using the VGG16 convolutional neural network with transfer learning. The study employs the publicly available RGB Arabic Alphabets Sign Language Dataset, comprising 7,856 annotated images across 31 Hijaiyah letters, collected under varied backgrounds and lighting conditions. The proposed model integrates pretrained ImageNet weights with a customized classification head, trained through a two-stage fine-tuning process with data augmentation. The model achieves 97.07% test accuracy, performing competitively against a ResNet-18 baseline (98.0%) while offering a simpler architecture suitable for resource-constrained deployments. Evaluation using precision, recall, F1-score, and confusion matrix shows consistently high performance, with minor misclassifications among visually similar letters. This work demonstrates a novel application of VGG16-based deep learning for ArSL recognition, contributing to inclusive religious education and accessibility technologies.
Downloads
References
S. Baghavathi Priya, P. V. R. S. Rao, and T. S. Madeswaran, “Sign to Speak: Real-time Recognition for Enhance Communication,” in Proceedings of the 3rd International Conference on Applied Artificial Intelligence and Computing, ICAAIC 2024, Institute of Electrical and Electronics Engineers Inc., 2024, pp. 871–876. doi: 10.1109/ICAAIC60222.2024.10575697.
D. Bisht, M. Kojage, M. Shukla, Y. Prakash Patil, and P. Bagade, “Smart Communication System Using Sign Language Interpretation,” in THE 31ST Conference of Fruct Asssociation, 2025. doi: 10.23919/FRUCT54823.2022.9770914.
M. H. Ismail, S. A. Dawwd, and F. H. Ali, “Static hand gesture recognition of Arabic sign language by using deep CNNs,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 24, no. 1, pp. 178–188, Oct. 2021, doi: 10.11591/ijeecs.v24.i1.pp178-188.
V. Kandukuri, S. R. Gundedi, V. Kamble, and V. Satpute, “Deaf and Mute Sign Language Translator on Static Alphabets Gestures using MobileNet,” in 2023 2nd International Conference on Paradigm Shifts in Communications Embedded Systems, Machine Learning and Signal Processing, PCEMS 2023, Institute of Electrical and Electronics Engineers Inc., 2023. doi: 10.1109/PCEMS58491.2023.10136074.
P. Jayadharshini, L. Krishnasamy, S. Santhiya, K. U. Harsine, P. Geetha, and B. Logu, “Enhanced Accessibility for Recognizing Indian and American Sign Language through Deep Learning Techniques,” in 2024 IEEE 4th International Conference on ICT in Business Industry and Government, ICTBIG 2024, Institute of Electrical and Electronics Engineers Inc., 2024. doi: 10.1109/ICTBIG64922.2024.10911064.
A. N. Handayani, M. I. Akbar, H. Ar-Rosyid, M. Ilham, R. A. Asmara, and O. Fukuda, “Design of SIBI Sign Language Recognition Using Artificial Neural Network Backpropagation,” in 2022 2nd International Conference on Intelligent Cybernetics Technology & Applications (ICICyTA), 2022, pp. 192–197. doi: 10.1109/ICICyTA57421.2022.10038205.
A. Deshpande, A. Shriwas, V. Deshmukh, and S. Kale, “Sign Language Recognition System using CNN,” in Proceedings of the International Conference on Intelligent and Innovative Technologies in Computing, Electrical and Electronics, ICIITCEE 2023, Institute of Electrical and Electronics Engineers Inc., 2023, pp. 906–911. doi: 10.1109/IITCEE57236.2023.10091051.
C. Author, A.-S. Khaled, and P. Wanda, “How to Stepping up Characters Recognition using CNN Algorithm?,” International Journal of Informatics and Computation (IJICOM), vol. 4, no. 2, 2022, doi: 10.35842/ijicom.
U. Fadlilah, A. K. Mahamad, and B. Handaga, “The Development of Android for Indonesian Sign Language Using Tensorflow Lite and CNN: An Initial Study,” in Journal of Physics: Conference Series, IOP Publishing Ltd, Apr. 2021. doi: 10.1088/1742-6596/1858/1/012085.
A. Rakhmadi and A. Yudhana, “A Study of Worldwide Patterns in Alphabet Sign Language Recognition Using Convolutional and Recurrent Neural Networks,” Jurnal Teknik Informatika (JUTIF), vol. 6, no. 1, pp. 187–204, Feb. 2025, doi: 10.52436/1.jutif.2025.6.1.4202.
A. Khan et al., “Deep Learning Approaches for Continuous Sign Language Recognition: A Comprehensive Review,” 2025, Institute of Electrical and Electronics Engineers Inc. doi: 10.1109/ACCESS.2025.3554046.
B. Alsharif, A. S. Altaher, A. Altaher, M. Ilyas, and E. Alalwany, “Deep Learning Technology to Recognize American Sign Language Alphabet,” Sensors, vol. 23, no. 18, Sep. 2023, doi: 10.3390/s23187970.
P. Adla, P. Pola, P. S. Kiran, Sh. Kumar, and Pitchai R, “Lightweight American Sign Language and Gesture Recognition using YOLOv8,” in 15th IEEE International Conference on Computational Intelligence and Communication Networks, IEEE, 2023.
B. Sundar and T. Bagyammal, “American Sign Language Recognition for Alphabets Using MediaPipe and LSTM,” in Procedia Computer Science, Elsevier B.V., 2022, pp. 642–651. doi: 10.1016/j.procs.2022.12.066.
E. A. Wijaya, S. B. Meliala, M. F. Hidayat, and I. A. Iswanto, “Sign Language Translator for SIBI,” in 2024 6th International Conference on Cybernetics and Intelligent System, ICORIS 2024, Institute of Electrical and Electronics Engineers Inc., 2024. doi: 10.1109/ICORIS63540.2024.10903922.
I. D. M. B. A. Darmawan et al., “Advancing Total Communication in SIBI: A Proposed Conceptual Framework for Sign Language Translation,” in 2023 International Conference on Smart-Green Technology in Electrical and Information Systems (ICSGTEIS), 2023, pp. 23–28. doi: 10.1109/ICSGTEIS60500.2023.10424020.
M. Arthur Limantara and D. Tristianto, “SIBI Alphabet Detection System Based on Convolutional Neural Network (CNN) Method as Learning Media,” Internet of Things and Artificial Intelligence Journal, vol. 4, no. 1, pp. 143–161, Mar. 2024, doi: 10.31763/iota.v4i1.716.
U. Fadlilah et al., “Android Application Prototype of Basic BISINDO Introduction and Practice for General People,” Sep. 2022.
M. B. Sasongko and N. Usman, “Analysis of The Quran Isyarat Learning Management at The Magelang Deaf Education Foundation,” Al Ulya, vol. 10, no. 01, pp. 20–40, 2025, doi: 10.32665/alulya.v10i1.3559.
B. Pamungkas, R. Wahab, and S. Suwarjo, “Teaching of the Quran and Hadiths Using Sign Language to Islamic Boarding School Students with Hearing Impairment,” International Journal of Learning, Teaching and Educational Research, vol. 22, no. 5, pp. 227–242, May 2023, doi: 10.26803/ijlter.22.5.11.
N. Irfan, P. Pakistan, and S. Seerat Hassan, “Perception of Teachers about Challenges Faced by Deaf Students in Learning Holy Quran (Muslim’s Religious Book) Faisal Amjad,” Research Journal (MRJ), vol. 5, 2024.
I. Dzulkifli, “Quranic Education for Deaf Students in Malaysia; Implementation and Challenges,” Nadwa: Jurnal Pendidikan Islam, 2022, doi: 10.21580/nw.2022.16.1.10823.
A. Alnuaim, M. Zakariah, W. A. Hatamleh, H. Tarazi, V. Tripathi, and E. T. Amoatey, “Human-Computer Interaction with Hand Gesture Recognition Using ResNet and MobileNet,” Comput Intell Neurosci, vol. 2022, 2022, doi: 10.1155/2022/8777355.
X. Han, F. Lu, and G. Tian, “Sign Language Recognition Based on Lightweight 3D MobileNet-v2 and Knowledge Distillation,” in ICETIS 2022, Harbin, Jan. 2022.
M. D. Meitantya, C. A. Sari, E. H. Rachmawanto, and R. R. Ali, “VGG-16 Architecture on CNN for American Sign Language Classification,” Jurnal Teknik Informatika (Jutif), vol. 5, no. 4, pp. 1165–1171, Jul. 2024, doi: 10.52436/1.jutif.2024.5.4.2160.
A. M. J. Al Moustafa et al., “Arabic Sign Language Recognition Systems: A Systematic Review,” Indian Journal of Computer Science and Engineering (IJCSE), vol. 15, 2024, doi: 10.21817/indjcse/2024/v15i1/241501008.
M. Al-Barham, A. A. Sa’aleek, M. Al-Odat, G. Hamad, M. Al-Yaman, and A. Elnagar, “Arabic Sign Language Recognition Using Deep Learning Models,” in 2022 13th International Conference on Information and Communication Systems, ICICS 2022, Institute of Electrical and Electronics Engineers Inc., 2022, pp. 226–231. doi: 10.1109/ICICS55353.2022.9811162.
M. Al-Barham et al., “RGB Arabic Alphabets Sign Language Dataset,” Jan. 2023, [Online]. Available: http://arxiv.org/abs/2301.11932
A. M. Farouk et al., “A New Approach for Arabic Sign Language Recognition (ArSLR),” in NILES 2024 - 6th Novel Intelligent and Leading Emerging Sciences Conference, Proceedings, Institute of Electrical and Electronics Engineers Inc., 2024, pp. 545–550. doi: 10.1109/NILES63360.2024.10753193.
B. Y. Al-Khuraym and M. M. Ben Ismail, “Arabic Sign Language Recognition using Lightweight CNN-based Architecture,” International Journal of Advanced Computer Science and Applications, vol. 13, no. 4, pp. 319–328, 2022, doi: 10.14569/IJACSA.2022.0130438.
M. H. Ismail, S. A. Dawwd, and F. H. Ali, “Arabic Sign Language Detection Using Deep Learning Based Pose Estimation,” in Proceedings of 2021 2nd Information Technology to Enhance E-Learning and other Application Conference, IT-ELA 2021, Institute of Electrical and Electronics Engineers Inc., 2021, pp. 161–166. doi: 10.1109/IT-ELA52201.2021.9773404.
M. R. Kale, C. Kaur, I. Kumar, H. S. Mahdi, V. Uprikar, and V. Sankari, “Development of an Arabic Sign Language Recognition System Utilizing Deep Convolutional Neural Network,” in 2025 5th International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies, ICAECT 2025, Institute of Electrical and Electronics Engineers Inc., 2025. doi: 10.1109/ICAECT63952.2025.10958963.
A. Mohan, D. Mohan, S. Vats, V. Sharma, and V. Kukreja, “Classification of Sign Language Gestures using CNN with Adam Optimizer,” in 2024 2nd International Conference on Disruptive Technologies, ICDT 2024, Institute of Electrical and Electronics Engineers Inc., 2024, pp. 430–433. doi: 10.1109/ICDT61202.2024.10489158.
D. O. Melinte and L. Vladareanu, “Facial Expressions Recognition for Human–Robot Interaction Using Deep Convolutional Neural Networks with Rectified Adam Optimizer,” Sensors, vol. 20, no. 8, 2020, doi: 10.3390/s20082393.
O. G. Ajayi and J. Ashi, “Effect of varying training epochs of a Faster Region-Based Convolutional Neural Network on the Accuracy of an Automatic Weed Classification Scheme,” Smart Agricultural Technology, vol. 3, Feb. 2023, doi: 10.1016/j.atech.2022.100128.
S. G. S. Kumar and J. Abbass, “Enhancing Sign Language Communication: Advanced Gesture Recognition Models for Indian Sign Language,” in 2025 International Research Conference on Smart Computing and Systems Engineering (SCSE), IEEE, Apr. 2025, pp. 1–6. doi: 10.1109/SCSE65633.2025.11031017.
S. A. Nivishna Shree, D. Sasikala, and S. A. Nehaa Shree, “Performance Analysis of Sign Language Recognition using Deep Neural Network Architecture,” in Proceedings - 4th International Conference on Smart Technologies, Communication and Robotics 2025, STCR 2025, Institute of Electrical and Electronics Engineers Inc., 2025. doi: 10.1109/STCR62650.2025.11020537.
M.-U.-K. Rico, “Performance Analysis of CNN Model for Image Classification with Intel OpenVINO on CPU and GPU,” Thesis, University of Windsor, Windsor, Ontario, Canada, 2023.
S. Kurundkar, A. Joshi, A. Thaploo, S. Auti, and A. Awalgaonkar, “Real-Time Sign Language Detection,” in ViTECoN 2023 - 2nd IEEE International Conference on Vision Towards Emerging Trends in Communication and Networking Technologies, Proceedings, Institute of Electrical and Electronics Engineers Inc., 2023. doi: 10.1109/ViTECoN58111.2023.10157784.
P. A. Prabha, M. Daanish, N. Kumar, and N. Kumaar, “Deep-SignSpeak: Deep Learning based Sign Language Recognition and Regional Language Translation,” in 2024 1st International Conference on Advanced Computing and Emerging Technologies, ACET 2024, Institute of Electrical and Electronics Engineers Inc., 2024. doi: 10.1109/ACET61898.2024.10730664.
V. K. Sambhav and R. Rajmohan, “Automated CNN model for Indian Sign Language Gesture Recognition,” in Proceedings of the 2024 10th International Conference on Communication and Signal Processing, ICCSP 2024, Institute of Electrical and Electronics Engineers Inc., 2024, pp. 1483–1487. doi: 10.1109/ICCSP60870.2024.10544149.
S. A. Shaba, S. I. Shroddha, M. J. Hossain, and M. F. Monir, “NeuralGesture Communication: Translating one Sign Language to Another Sign Language Using Deep Learning Model and gTTs,” in IEEE Vehicular Technology Conference, Institute of Electrical and Electronics Engineers Inc., 2024. doi: 10.1109/VTC2024-Spring62846.2024.10683444.
M. Soundarya, M. Yazhini, N. Thirumala Sree, N. Sornamalaya, and C. Vinitha, “Sign Language Recognition Using Machine Learning,” in Proceedings - 3rd International Conference on Advances in Computing, Communication and Applied Informatics, ACCAI 2024, Institute of Electrical and Electronics Engineers Inc., 2024. doi: 10.1109/ACCAI61061.2024.10602025.
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Aris Rakhmadi, Anton Yudhana, Sunardi
This work is licensed under a Creative Commons Attribution 4.0 International License.
