Alphabet Gesture Classification of Indonesian Sign Language Using Convolutional Neural Networks

Yanuar  Gideon Simalango; Anindita Septiarini; Masna Wati; Hamdani Hamdani; Rajiansyah Rajiansyah

doi:10.52436/1.jutif.2026.7.1.5240

Authors

Yanuar Gideon Simalango Department of Informatics, Mulawarman University, Indonesia
Anindita Septiarini Department of Informatics, Mulawarman University, Indonesia
Masna Wati Department of Informatics, Mulawarman University, Indonesia
Hamdani Department of Informatics, Mulawarman University, Indonesia
Rajiansyah Department of Informatics, Mulawarman University, Indonesia

DOI:

https://doi.org/10.52436/1.jutif.2026.7.1.5240

Keywords:

Augmentation, BISINDO, Preprocessing, Recognition, Translation

Abstract

Indonesian Sign Language (BISINDO) serves as a communication medium for deaf individuals to engage with their environment. Alphabet gestures in BISINDO play a crucial role in the formation of words and sentences. Nonetheless, the automatic recognition of BISINDO alphabet movements remains a difficulty in the advancement of accessible technology. This research intends to categorize BISINDO alphabet gestures via the Convolutional Neural Network (CNN) model. The CNN approach was used due to its proficiency in recognizing visual patterns and images. The dataset comprises BISINDO alphabet gesture photos captured from diverse perspectives and lighting conditions. The data processing procedure encompasses pre-processing phases, including picture normalization, data augmentation, and the segmentation of the dataset into training, validation, and test subsets. The constructed CNN model has multiple convolutional and pooling layers to thoroughly extract visual characteristics. The study's results indicate that the CNN model can classify BISINDO alphabet gestures with a high accuracy of 90% on the test data. This model's deployment is anticipated to aid in the creation of automatic sign language translation programs, hence enhancing communication between the deaf community and the general populace. This study demonstrates the potential of CNN models to support the development of inclusive communication technologies for the hearing impaired in Indonesia, particularly for under-researched sign languages like BISINDO.

Downloads

Download data is not yet available.

References

S. Singh, A. Yadav, And G. C. Nandi, “Real-Time Static Hand Gesture Recognition Using Convolutional Neural Networks,” Computers & Electrical Engineering, Vol. 92, pp. 107109, 2021.

M. Tran and H. Le, “Vietnamese Sign Language Recognition Using Convolutional Neural Networks and Data Augmentation,” International Journal of Advanced Computer Science and Applications, Vol. 12, No. 6, pp. 476–482, 2021.

T. Bui, D. Nguyen, And N. Vo, “Skin Segmentation Preprocessing for Vietnamese Sign Language Recognition Using CNN,” Multimedia Tools and Applications, Vol. 82, No. 1, pp. 1097–1113, 2023.

A. Chatterjee, M. Singh, And R. Jain, “Transfer Learning with Mobilenetv2 for Lightweight Indian Sign Language Recognition,” Procedia Computer Science, Vol. 171, pp. 1231–1238, 2020.

M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, And L. C. Chen, “Mobilenetv2: Inverted Residuals and Linear Bottlenecks,” Ieee Transactions on Pattern Analysis and Machine Intelligence, Vol. 42, No. 5, pp. 1257–1272, 2020.

K. Nakamura and Y. Tanaka, "Application of EfficientNet for hand sign classification," Signal Processing: Image Communication, vol. 96, pp. 116344, 2021, Doi: 10.1016/j.image.2021.116344.

A. Sharma and S. Kumar, "Advances in CNN architectures for image recognition," International Journal of Computer Vision, vol. 129, no. 6, pp. 2023-2041, 2023, Doi: 10.1007/s11263-023-01620-4.

P. Kumar et al., "An overview of deep learning techniques in gesture recognition," International Journal of Pattern Recognition and Artificial Intelligence, vol. 36, no. 5, pp. 2256002, 2022, Doi: 10.1142/S0218001422560027.

T. Wang, "Recent Trends in Deep Learning-Based Sign Language Translation," IEEE Access, vol. 10, pp. 122034-122047, 2022, Doi: 10.1109/ACCESS.2022.3224567.

Y. Zhao, Y. Lin, And J. Wu, “Fast and Accurate CNN-Based Sign Language Recognition Using Real-Time Video Stream,” Neural Computing and Applications, Vol. 33, pp. 7927–7942, 2021.

R. Elakkiya, N. Gopalakrishnan, And R. Prabu, “Sequence Modelling of Sign Language Recognition Using CNN-LSTM Hybrid Architecture,” Multimedia Tools and Applications, Vol. 82, pp. 17935–17958, 2023.

K. Goyal, R. Singla, And T. Choudhury, “Deep Learning for Sign Language Recognition: Current Trends and Challenges,” Procedia Computer Science, Vol. 167, pp. 2481– 2490, 2020.

F. W. D. S. U. R. Jumaryadi, "Implementasi Convolutional Neural Network Dalam Klasifikasi Citra," Jurnal Teknik Informatika, vol. 6, no. 6, pp. 1530–1537, Dec. 2025.

J. Zhang et al., "Sign Language Recognition with Deep Learning: A Review," IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 2, pp. 456-472, 2023, Doi: 10.1109/TNNLS.2022.3201476.

O. Alzubaidi Et Al., “Review of Deep Learning: Concepts, CNN Architectures, Challenges, Applications, Future Directions,” Journal of Big Data, Vol. 8, No. 1, Pp. 1–74, 2021.

Z. H. Salsabila, R. R. Nurmalasari and L. Kamelia, "Indonesian Sign Language Translation System Using ResNet-50 Architecture-Based Convolutional Neural Network," 2024 10th International Conference on Wireless and Telematics (ICWT), Batam, Indonesia, 2024, pp. 1-5, doi: 10.1109/ICWT62080.2024.10674686.

M. M. Alnfiai, “Deep Learning-Based Sign Language Recognition for Hearing and Speaking Impaired People,” Intelligent Automation & Soft Computing, vol. 36, no. 2, pp. 1653–1669, 2023.

A. N. Sihananto, E. M. Safitri, Y. Maulana, F. Fakhruddin, and M. E. Yudistira, "Indonesian Sign Language Image Detection Using Convolutional Neural Network (CNN) Method," Inspiration: Jurnal Teknologi Informasi dan Komunikasi, vol. 13, no. 1, pp. 13–21, 2023.

L. Chen, Y. Li, And H. Wang, “Improving Hand Gesture Recognition Through Enhanced Image Preprocessing for CNN Models,” Journal of Visual Communication And Image Representation, Vol. 94, 103751, 2023.

A. Lee and R. White, "Transfer learning in small-scale sign language datasets," Journal of Machine Learning Research, vol. 23, no. 42, pp. 1-20, 2022.

S. Li and H. Fu, "Lightweight CNNs for Gesture Recognition on Mobile Devices," Pattern Recognition Letters, vol. 160, pp. 30-38, 2022, Doi: 10.1016/j.patrec.2022.06.012.

A. Rahim, R. Khan, and S. Abbas, “Robust Sign Language Recognition Using CNN With Data Augmentation,” Journal of Ambient Intelligence and Humanized Computing, Vol. 13, No. 5, pp. 2395–2406, 2022.

M. Lopez et al., "Data augmentation for hand gesture recognition using CNNs," Journal of Visual Communication and Image Representation, vol. 88, pp. 103539, 2022, Doi: 10.1016/j.jvcir.2022.103539.

R. Hao, K. Namdar, L. Liu, M. A. Haider, And F. Khalvati, "A Comprehensive Study of Data Augmentation Strategies for Prostate Cancer Detection in Diffusion-Weighted MRI Using Convolutional Neural Networks," Journal of Digital Imaging, Vol. 34, Pp. 862–876, 2021, Doi: 10.1007/S10278-021-00478-7.

N. Patel and D. Shah, "Optimizing CNN Models for Sign Language Recognition in Low-Resource Settings," Computer Vision and Image Understanding, vol. 220, pp. 103451, 2022, Doi: 10.1016/j.cviu.2022.103451.