PATTERN CLASSIFICATION SIGN LANGUAGE USING FEATURES DESCRIPTORS AND MACHINE LEARNING

Nurhadi Nurhadi; Eko Arip Winanto; Rahaini Mohd  Said; Jasmir Jasmir; Lasmedi  Afuan

doi:10.52436/1.jutif.2024.5.2.1228

Nurhadi Informatics, Faculty of Computer Science, Universits Dinamika Bangsa, Indonesia
Eko Arip Winanto Computer Engineering, Faculty of Computer Science, Universits Dinamika Bangsa, Indonesia
Rahaini Mohd Said Faculty of technology electric and electronic, Universiti Teknikal Malaysia Melaka, Malaysia
Jasmir Computer Engineering, Faculty of Computer Science, Universits Dinamika Bangsa, Indonesia
Lasmedi Afuan Informatics, Faculty of Computer Science, Universitas Jenderal Soedirman, Indonesia

DOI: https://doi.org/10.52436/1.jutif.2024.5.2.1228

Keywords: Color histogram, Fingerspelling, Haralick texture, Hu Moment, Logistic regression segmentation

Abstract

Sign language is way of communication for the deaf and speech impaired. In Indonesia, the utilization of a standardized language involves the incorporation of American Sign Language (ASL). ASL is employed for various communication needs, ranging from basic alphanumeric fingerspelling (A-Z and numbers) to the more complex SIBI form (comprising gesture vocabulary) in everyday interactions as well as formal contexts. This surge in the digitization of sign language underscores the ongoing advancements in research and development. The challenge in this research lies in the ability to recognize American Sign Language (ASL) with diverse intensities and invariant backgrounds. Therefore, the study emphasis is on proposing a suitable segmentation method comparison for multi-intensity ASL cases. Subsequently, global feature descriptor methods, including Color Histogram, Hu Moments, and Haralick Texture techniques, are applied for feature extraction. The result of the Logistic Regression method versus the supervised Random Forest checks accuracy and suitability in identifying ASL fingerspelling. The findings of this research is predictive value of logistic regression is 48%, with class Y having the highest precision (0.86), class V having the lowest accuracy (0.16), and class L having the highest recall (0.73). The maximum precision in classes B, F, H, I, K, Y, and Z is 1.00, and the lowest in class U is 0.58, while the highest recall is in class G, which is 1.00. The lowest is in class V, while the predictive value from the random forest is 86 percent. Class H has the greatest f1 score (0.99), while class U has the lowest f1 score (0.64). The Random Forest method outperforms the two methods suggested in the paper, according to the comparison.

Downloads

Download data is not yet available.

References

A. Khan, I. Alam, M. F. Khan, I. Ahmed, Z. U. Islam, and M. Azam, “A comprehensive analysis of adaptive image restoration techniques in the presence of different noise models,” Found. Univ. J. Eng. Appl. Sci. (HEC Recognized Y Categ. ISSN 2706-7351), vol. 1, no. 2, pp. 6–13, 2020.

H. W. A. Internet, “a S Urvey of I Ntegrating,” Communications, vol. 6, no. 3, pp. 14–30, 2007.

H. ZainEldin et al., “Brain Tumor Detection and Classification Using Deep Learning and Sine-Cosine Fitness Grey Wolf Optimization,” Bioengineering, vol. 10, no. 1, pp. 1–19, 2023, doi: 10.3390/bioengineering10010018.

M. Sheykhmousa, M. Mahdianpari, H. Ghanbari, F. Mohammadimanesh, P. Ghamisi, and S. Homayouni, “Support Vector Machine Versus Random Forest for Remote Sensing Image Classification: A Meta-Analysis and Systematic Review,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 13, pp. 6308–6325, 2020, doi: 10.1109/JSTARS.2020.3026724.

A. Latif et al., “Content-based image retrieval and feature extraction: A comprehensive review,” Math. Probl. Eng., vol. 2019, 2019, doi: 10.1155/2019/9658350.

M. Keivani, J. Mazloum, E. Sedaghatfar, and M. B. Tavakoli, “Automated analysis of leaf shape, texture, and color features for plant classification,” Trait. du Signal, vol. 37, no. 1, pp. 17–28, 2020, doi: 10.18280/ts.370103.

A. Indian and K. Bhatia, “Offline Handwritten Hindi Numerals Recongnition using Zernika Moments,” Int. J. Tomogr. Simul., vol. 32, no. 2, 2019.

G. Ramesh, J. Logeshwaran, J. Gowri, and A. Mathew, “The management and reduction of digital noise in video image processing by using transmission based noise elimination scheme” ICTACT J. IMAGE VIDEO Process., vol. 9102, no. November, pp. 2797–2801, 2022, doi: 10.21917/ijivp.2022.0398.

P. H. Gonçalves Dias Diniz, “Chemometrics-assisted color histogram-based analytical systems,” J. Chemom., vol. 34, no. 12, pp. 1–27, 2020, doi: 10.1002/cem.3242.

Z. AlQadi, “A Highly Secure and Accurate Method for RGB Image Encryption‏,” Int. J. Comput. Sci. Mob. Comput., vol. 9, no. 1, pp. 12–21, 2020

P. K. Athira, C. J. Sruthi, and A. Lijiya, “A Signer Independent Sign Language Recognition with Co-articulation Elimination from Live Videos: An Indian Scenario,” J. King Saud Univ. - Comput. Inf. Sci., vol. 34, no. 3, pp. 771–781, 2022, doi: 10.1016/j.jksuci.2019.05.002.

A. Tyagi and S. Bansal, “Feature Extraction Technique for Vision-Based Indian Sign Language Recognition System: A Review,” in Computational Methods and Data Engineering, 2021, pp. 39–53.

W. Liu, S. Zeng, G. Wu, H. Li, and F. Chen, “Rice seed purity identification technology using hyperspectral image with lasso logistic regression model,” Sensors, vol. 21, no. 13, 2021, doi: 10.3390/s21134384.

M. M. Chen and M. C. Chen, “Modeling road accident severity with comparisons of logistic regression, decision tree and random forest,” Inf., vol. 11, no. 5, 2020, doi: 10.3390/INFO11050270.

T. Lan, H. Hu, C. Jiang, G. Yang, and Z. Zhao, “A comparative study of decision tree, random forest, and convolutional neural network for spread-F identification,” Adv. Sp. Res., vol. 65, no. 8, pp. 2052–2061, 2020, doi: 10.1016/j.asr.2020.01.036.

M. Subramanian, V. Lingamuthu, C. Venkatesan, and S. Perumal, “Content-Based Image Retrieval Using Colour, Gray, Advanced Texture, Shape Features, and Random Forest Classifier with Optimized Particle Swarm Optimization,” Int. J. Biomed. Imaging, vol. 2022, 2022, doi: 10.1155/2022/3211793.