Hand Image-Based BISINDO Alphabet Classification Utilizing Convolutional Neural Network on Android Application

Saniyyah Intan  Salsabiila; Nataniel Dengen; Joan Angelina Widians

doi:10.52436/1.jutif.2026.7.4.6294

Authors

Saniyyah Intan Salsabiila Department of Informatics, Mulawarman University, Indonesia
Nataniel Dengen Department of Informatics, Mulawarman University, Indonesia
Joan Angelina Widians Master's Program of Informatics, Mulawarman University, Indonesia

DOI:

https://doi.org/10.52436/1.jutif.2026.7.4.6294

Keywords:

Artificial Intelegence, BISINDO, Convolutional Neural Network, Digital image processing, Sign Language

Abstract

Communication barriers between people with hearing impairments and the general public remain a significant challenge in everyday interactions. BISINDO is a widely used sign language in Indonesia’s deaf community; however, public understanding of sign language remains limited. This study aims to develop a hand image-based BISINDO alphabet classification system using a Convolutional Neural Network model and implement the trained model in an Android application. The public dataset used consists of 10,686 BISINDO alphabet letter images representing 26 classes from A to Z, obtained from Kaggle. All images were preprocessed, including resizing, normalization, and data augmentation to increase the diversity of the training data. The model was trained and evaluated using accuracy, precision, recall, and F1-score metrics. The evaluation results showed that the model achieved a validation accuracy of 94.21%, precision of 94.38%, recall of 94.20%, and F1-score of 94.21%. The trained model was then integrated into an Android application named IsyaratKu using the Flutter framework and the Dart programming language. The implementation results showed that the application successfully recognized the letters of the BISINDO alphabet through camera and gallery input and performed effectively on Android devices. These results indicate that the CNN can classify the 26 BISINDO alphabet classes with validation accuracy above 94% and can be deployed in an Android application for offline alphabet classification.

Downloads

Download data is not yet available.

References

A. Nugroho, R. Setiawan, A. Harris, and Beny, “Deteksi Bahasa Isyarat Bisindo Menggunakan Metode Machine Learning,” Jurnal Ilmiah Sistem Informasi, Teknologi Informasi dan Sistem Komputer, vol. 18, no. 2, pp. 152–158, Oct. 2023, doi: 10.33998/processor.2023.18.2.1308.

R. Mandasari and S. Winduwati, “Upaya Public Relations Pusbisindo dalam Mengampanyekan Penggunaan Bahasa Isyarat Indonesia di Kalangan Masyarakat,” vol. 6, no. 2, pp. 355–361, Oct. 2022, doi: https://doi.org/10.24912/pr.v6i2.15572.

F. I. Hadinata and S. A. Sanjaya, “BISINDO Sign Language Recognition: A Systematic Literature Review of Deep Learning Techniques for Image Processing,” Indonesian Journal of Computer Science Attribution, vol. 12, no. 6, pp. 3281–3294, 2023, doi: https://doi.org/10.33022/ijcs.v12i6.3539.

Kementerian Kesehatan Republik Indonesia, “Survei Kesehatan Indonesia (SKI) 2023 Dalam Angka,” Jakarta, 2024. Accessed: May 22, 2026. [Online]. Available: https://www.badankebijakan.kemkes.go.id/ski-2023-dalam-angka/

A. M. Ambarak and A. Z. Falani, “Pengembangan Aplikasi Bahasa Isyarat Indonesia Berbasis Realtime Video Menggunakan Model Machine Learning,” JIKA (Jurnal of Informatics), vol. 7, no. 1, pp. 89–96, Jan. 2023, doi: 10.31000/jika.v7i1.7277.

A. Kinanti, M. A. Afandi, I. Permatasari, and N. Y. Tarigan, “Deteksi Objek Bahasa Isyarat Alfabet BISINDO Menggunakan Deep Learning dan Arsitektur YOLO,” Techno.COM, vol. 23, no. 2, pp. 409–417, May 2024, doi: https://doi.org/10.62411/tc.v23i2.9889.

H. A. D. Rimbawa, M. I. AlFatrah, M. F. Rais, C. Z. Noorta, and A. Atturoybi, “Artificial intelligence-based hand gesture recognition for sign language interpretation,” Jurnal Mandiri IT, vol. 14, no. 1, pp. 76–86, Jul. 2025, doi: https://doi.org/10.35335/mandiri.v14i1.395.

Y. Brianorman and R. Munir, “Perbandingan Pre-Trained CNN: Klasifikasi Pengenalan Bahasa Isyarat Huruf Hijaiyah,” Jurnal Sistem Informasi Bisnis, vol. 13, no. 1, pp. 52–59, Jul. 2023, doi: 10.21456/vol13iss1pp52-59.

M. A. R. Mahardika, N. Yudistira, and A. Ridok, “Sistem Rekognisi Citra Digital Bahasa Isyarat Menggunakan Convolutional Neural Network dan Spatial Transformer,” Jurnal Teknologi Informasi dan Ilmu Komputer, vol. 11, no. 6, pp. 1159–1168, Dec. 2024, doi: 10.25126/jtiik.2023118098.

R. Soekarta, M. Yusuf, M. F. Hasa, and N. A. Basri, “Implementasi Deep Learning Untuk Deteksi Jenis Obat Menggunakan Algoritma CNN Berbasis Website,” JIKA (Jurnal of Informatics), vol. 7, no. 4, pp. 455–464, Nov. 2023, doi: 10.31000/jika.v7i4.9751.

L. Arisandi and B. Satya, “Sistem Klarifikasi Bahasa Isyarat Indonesia (Bisindo) Dengan Menggunakan Algoritma Convolutional Neural Network,” Jurnal Sistem Cerdas, vol. 5, no. 3, pp. 135–146, 2022, doi: https://doi.org/10.37396/jsc.v5i3.262.

I. G. A. O. Aryananda and F. Samopa, “Comparison of the Accuracy of the Bahasa Isyarat Indonesia (BISINDO) Detection System Using CNN and RNN Algorithm for Implementation on Android,” MALCOM: Indonesian Journal of Machine Learning and Computer Science, vol. 4, no. 3, pp. 1111–1119, Jul. 2024, doi: 10.57152/malcom.v4i3.1465.

E. L. Kelana, M. R. A. Prasetya, Mambang, and M. Zulfadhilah, “Integrating the CNN Model with the Web for Indonesian Sign Language (BISINDO) Recognition,” Journal of Applied Informatics and Computing (JAIC), vol. 9, no. 3, pp. 883–896, Jun. 2025, doi: https://doi.org/10.30871/jaic.v9i3.9345.

W. Z. Putri, I. Salamah, and E. Hesti, “Penerapan Convolutional Neural Network Pada Aplikasi Terjemahan Bahasa Isyarat Indonesia Berbasis Android,” Decode: Jurnal Pendidikan Teknologi Informasi, vol. 5, no. 2, pp. 641–653, Jul. 2025, doi: 10.51454/decode.v5i2.1226.

A. N. Sihananto, E. M. Safitri, Y. Maulana, F. Fakhruddin, and M. E. Yudistira, “Indonesian Sign Language Image Detection Using Convolutional Neural Network (CNN) Method,” Jurnal Teknologi Informasi dan Komunikasi, vol. 13, no. 1, pp. 13–21, Jun. 2023, doi: https://doi.org/10.35585/inspir.v13i1.37.

T. B. Sasongko, Haryoko, and A. Amrullah, “Analisis Efek Augmentasi Dataset dan Fine Tune pada Algoritma Pre-Trained Convolutional Neural Network (CNN),” Jurnal Teknologi Informasi dan Ilmu Komputer (JTIIK), vol. 10, no. 4, pp. 763–768, Aug. 2023, doi: 10.25126/jtiik.2023106583.

R. Karnoto and I. A. Nisa, “Aplikasi Identifikasi Angka Jari 1-5 Menggunakan CNN Berbasis Pengolahan Citra,” in Seminar Nasional Teknologi dan Sains Tahun 2026, Kediri: Universitas Nusantara PGRI Kediri, Jan. 2026, pp. 852–857. doi: https://doi.org/10.29407/xegsf729.

R. Fahlevi and C. Rozikin, “Identifikasi Isyarat Tangan Bisindo Dengan Algoritma CNN Dan Transfer Learning Menggunakan Mobilenetv2,” JATI (Jurnal Mahasiswa Teknik Informatika), vol. 9, no. 4, pp. 6592–6597, Aug. 2025, doi: 10.36040/jati.v9i4.14095.

R. Z. Fadillah, A. Irawan, and M. Susanty, “Model Penerjemah Bahasa Isyarat Indonesia (BISINDO) Menggunakan Pendekatan Transfer Learning,” PETIR: Jurnal Pengkajian dan Penerapan Teknik Informatika, vol. 15, no. 1, pp. 1–9, Mar. 2022, doi: 10.33322/petir.v15i1.1289.

M. A. Firdaus, T. Dewangga, D. R. Indah, R. I. Heroza, J. A. Kusjadi, E. Martin, et al., “Precision-Optimized CNN for Indonesian Sign Language Recognition on Mobile Devices,” Jurnal Sains dan Teknologi, vol. 14, no. 3, pp. 425–436, 2025, doi: 10.23887/jstundiksha.v14i3.101617.

A. G. Arrozaq and Latipah, “Implementasi Computer Vision untuk Terjemahkan Abjad Bahasa Isyarat SIBI pada Aplikasi Android,” Jurnal Teknologi Dan Sistem Informasi Bisnis, vol. 6, no. 2, pp. 304–313, Apr. 2024, doi: https://doi.org/10.47233/jteksis.v6i2.1260.

R. Aryanto, M. A. Rosid, and S. Busono, “Penerapan Deep Learning untuk Pengenalan Tulisan Tangan Bahasa Aksara Lota Ende dengan Menggunakan Metode Convolutional Neural Networks,” Jurnal Informasi dan Teknologi, vol. 5, no. 1, pp. 258–264, May 2023, doi: 10.37034/jidt.v5i1.313.

X. Liu, Y. Jiang, Y. Cui, J. Yuan, and X. Fang, “Deep learning in single-molecule imaging and analysis: recent advances and prospects,” Chem. Sci., vol. 13, pp. 11964–11980, Sep. 2022, doi: 10.1039/d2sc02443h.

M. N. I. Muhlashin and A. Stefanie, “Klasifikasi Penyakit Mata Berdasarkan Citra Fundus Menggunakan Yolo V8,” JATI (Jurnal Mahasiswa Teknik Informatika), vol. 7, no. 2, pp. 1363–1368, Apr. 2023, doi: 10.36040/jati.v7i2.6927.

A. Riyandi and S. ‘Uyun, “Improvement of Handwriting Javascraft Image Quality and Segmentation with Closing Morphology and Adaptive Thresholding Methods,” Telematika: Jurnal Informatika dan Teknologi Informasi, vol. 19, no. 3, pp. 311–322, Oct. 2022, doi: 10.31515/telematika.v19i3.7564.

Sumiati, R. M. Firzatullah, V. Rosalina, and D. Karyaningsih, “Development of CNN and YOLO Models to Detect Smoking Violations of Cadets in the Environment Politeknik Transportasi SDP Palembang,” Journal of Internet Services and Information Security (JISIS), vol. 15, no. 1, pp. 107–129, Feb. 2025, doi: 10.58346/JISIS.2025.I1.008.

S. A. Adzkia and T. Arifin, “Perbandingan Arsitektur CNN Berbasis Transfer Learning untuk Klasifikasi pada BreastMNIST,” Jurnal Nasional Teknologi dan Sistem Informasi, vol. 11, no. 2, pp. 192–200, Sep. 2025, doi: 10.25077/teknosi.v11i2.2025.192-200.

H. Yun, E. Kim, D. M. Kim, H. W. Park, and M. B. G. Jun, “Machine Learning for Object Recognition in Manufacturing Applications,” Jan. 2023, SpringerOpen. doi: 10.1007/s12541-022-00764-6.

J. Zheng and M. Du, “Study on Tomato Disease Classification based on Leaf Image Recognition based on Deep Learning Technology,” (IJACSA) International Journal of Advanced Computer Science and Applications, vol. 14, no. 4, pp. 575–583, 2023, doi: https://doi.org/10.14569/IJACSA.2023.0140463.

P. Mavaie, L. Holder, and M. K. Skinner, “Hybrid deep learning approach to improve classification of low-volume high-dimensional data,” BMC Bioinformatics, vol. 24, pp. 1–20, 2023, doi: 10.1186/s12859-023-05557-w.

D. Parashar, S. Thakur, K. B. Raju, G. B. Madhavi, and K. Sharma, “A Deep Learning-Based Approach for Hand Sign Recognition Using CNN Architecture,” Revue d’Intelligence Artificielle, vol. 37, no. 4, pp. 937–943, Aug. 2023, doi: 10.18280/ria.370414.

A. Riyandi, T. Widodo, and S. Uyun, “Classification of Damaged Road Images Using the Convolutional Neural Network Method,” Jurnal Informatika dan Teknologi Informasi, vol. 19, no. 2, pp. 147–158, Jun. 2022, doi: 10.31515/telematika.v19i2.6460.

S. F. Ahmed, M. S. B. Alam, M. Hassan, M. R. Rozbu, T. Ishtiak, N. Rafa, et al., “Deep learning modelling techniques: current progress, applications, advantages, and challenges,” Artif. Intell. Rev., vol. 56, pp. 13521–13617, Apr. 2023, doi: 10.1007/s10462-023-10466-8.

X. Zhao, L. Wang, Y. Zhang, X. Han, M. Deveci, and M. Parmar, “A review of convolutional neural networks in computer vision,” Artif. Intell. Rev., vol. 57, Apr. 2024, doi: 10.1007/s10462-024-10721-6.

N. P. Sutrisna, R. A. Sahirah, K. S. S. Laksono, R. A. S. Permadhi, N. Nurannisa, S. S. Larasati, et al., “Deteksi Tingkat Kematangan Buah Pepaya Menggunakan Model Convolutional Neural Network,” Jurnal Teknologi Informasi dan Ilmu Komputer (JTIIK), vol. 11, no. 3, pp. 569–577, Jun. 2024, doi: 10.25126/jtiik.938119.

N. D. Alqahtani, B. Alzahrani, and M. S. Ramzan, “Detection of Dyslexia Through Images of Handwriting Using Hybrid AI Approach,” International Journal of Advanced Computer Science and Applications, vol. 14, no. 10, pp. 942–951, 2023, doi: https://doi.org/10.14569/IJACSA.2023.0141099.

Z. Y. Hamd, A. I. Alorainy, M. A. Alharbi, A. Hamdoun, A. Alkhedeiri, S. Alhegail, et al., “Deep learning-based automated bone age estimation for Saudi patients on hand radiograph images: a retrospective study,” BMC Med. Imaging, vol. 24, no. 1, 2024, doi: 10.1186/s12880-024-01378-2.

R. Sahay and R. Kaur, “Plant Recognition and its Care using TFLite, Teachable Machine,” Int. J. Res. Appl. Sci. Eng. Technol., vol. 11, no. 5, pp. 772–778, May 2023, doi: 10.22214/ijraset.2023.51367.

Y. N. Fuadah, S. Saidah, N. K. Sy, R. Magdalena, and I. D. Ubaidullah, “Glaucoma Classification Based on Fundus Images Processing with Convolutional Neural Network,” Jurnal Teknik Informatika (JUTIF), vol. 3, no. 3, pp. 717–722, Jun. 2022, doi: 10.20884/1.jutif.2022.3.3.276.

B. Nurhakim, D. Solihudin, D. Amalia, and I. Arelia, “Data Augmentation Techniques on the Accuracy of Fertile and Infertile Egg Classification Using Convolutional Neural Networks,” Jurnal Teknik Informatika (JUTIF), vol. 6, no. 5, pp. 3787–3799, Oct. 2025, doi: 10.52436/1.jutif.2025.6.5.5234.