OBJECT DETECTION OF INDONESIAN SIGN LANGUAGE SYSTEM USING YOLOV7 METHOD

Abstract

SIBI or Indonesian Sign Language System, a communication language for the deaf community in Indonesia. SIBI has the advantage of conveying information between individuals. SIBI integrates various hand signals to replace words in Indonesian, enabling effective and inclusive communication. SIBI still lacks educational programs in the community and identifying SIBI has become a major problem in facilitating communication for normal people with hearing impairments. The proposed solution in the development of SIBI detection is to utilize artificial intelligence (AI) technology and digital image processing. This program focuses on understanding the typical hand movements used in SIBI. So a program was created to detect hand language using the YOLOv7 architecture. This study aims to educate those who are not yet familiar with the SIBI hand object that will be detected., especially in the context of sign language recognition for singular pronouns. The research method used is data acquisition by collecting a dataset of 320 images, data annotation by labeling objects on the hand, image pre-processing with augmentation, resizing, and cropping, model training with 100 epochs on both pre-trained models (yolov7 and yolov7-x), and testing is done by detecting 20 images from each class category totaling 5. The dataset used for training 300 images and validation 20 images. The results of the yolov7 model accuracy value are mAP @ .5 of 99.5% and mAP @ .5: .95: of 90.5%. The accuracy of the yolov7-x model is mAP @ .5 99.6% and mAP @ .5: .95: of 75.8%. And the results of the test carried out with 20 images, out of 20 correct images only 18 and the accuracy value obtained is 90%.