A MACHINE LEARNING APPROACH TO EYE BLINK DETECTION IN LOW-LIGHT VIDEOS

Muhammad Furqan Rasyid; Muhammad  Rizal; Wilem  Musu; Muhammad Sabirin  Hadis

doi:10.52436/1.jutif.2023.4.3.1024

Muhammad Furqan Rasyid Department of Informatics Management, Dipa Makassar University, Indonesia
Muhammad Rizal Department of Informatics Engineering, Dipa Makassar University, Indonesia
Wilem Musu Department of Informatics Engineering, Dipa Makassar University, Indonesia
Muhammad Sabirin Hadis Department of Electrical Engineering and Computer Science, Kanazawa University, Kanazawa, Japan

DOI: https://doi.org/10.52436/1.jutif.2023.4.3.1024

Keywords: confusion matrix, eye blink detection, flashlight, low-light video, machine learning

Abstract

Inadequate lighting conditions can harm the accuracy of blink detection systems, which play a crucial role in fatigue detection technology, transportation and security applications. While some video capture devices are now equipped with flashlight technology to enhance lighting, users occasionally need to remember to activate this feature, resulting in slightly darker videos. Consequently, there is a pressing need to improve the performance of blink detection systems to detect eye accurately blinks in low light videos. This research proposes developing a machine learning-based blink detection system to see flashes in low-light videos. The Confusion matrix was conducted to evaluate the effectiveness of the proposed blink detection system. These tests involved 31 videos ranging from 5 to 10 seconds in duration. Involving male and female test subjects aged between 20 and 22. The accuracy of the proposed blink detection system was measured using the confusion matrix method. The results indicate that by leveraging a machine learning approach, the blink detection system achieved a remarkable accuracy of 100% in detecting blinks within low-light videos. However, this research necessitates further development to account for more complex and diverse real-life situations. Future studies could focus on developing more sophisticated algorithms and expanding the test subjects to improve the performance of the blink detection system in low light conditions. Such advancements would contribute to the practical application of the system in a broader range of scenarios, ultimately enhancing its effectiveness in fatigue detection technology.

Downloads

Download data is not yet available.

References

D. Kondratyuk et al., “MoViNets: Mobile Video Networks for Efficient Video Recognition,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA: IEEE, Jun. 2021, pp. 16015–16025. doi: 10.1109/CVPR46437.2021.01576.

A. Hadid and M. Pietikäinen, “Combining appearance and motion for face and gender recognition from videos,” Pattern Recognition, vol. 42, no. 11, pp. 2818–2827, Nov. 2009, doi: 10.1016/j.patcog.2009.02.011.

C. Dewi, R.-C. Chen, X. Jiang, and H. Yu, “Adjusting eye aspect ratio for strong eye blink detection based on facial landmarks,” PeerJ Comput. Sci., vol. 8, p. e943, Apr. 2022, doi: 10.7717/peerj-cs.943.

J. Cao et al., “Unsupervised Eye Blink Artifact Detection From EEG With Gaussian Mixture Model,” IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 8, pp. 2895–2905, Aug. 2021, doi: 10.1109/JBHI.2021.3057891.

M. Wang et al., “Multidimensional Feature Optimization Based Eye Blink Detection Under Epileptiform Discharges,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 30, pp. 905–914, 2022, doi: 10.1109/TNSRE.2022.3164126.

M. F. Rasyid, D. Imran, and A. A. Mahersatillah, “Prediksi penyebaran Sub Varian omicron di Indonesia menggunakan Machine Learning,” SISITI : Seminar Ilmiah Sistem Informasi dan Teknologi Informasi, vol. 11, no. 1, Art. no. 1, Aug. 2022, Accessed: Jan. 11, 2023. [Online]. Available: https://www.ejurnal.dipanegara.ac.id/index.php/sisiti/article/view/936

L. Zhang et al., “A review of machine learning in building load prediction,” Applied Energy, vol. 285, p. 116452, Mar. 2021, doi: 10.1016/j.apenergy.2021.116452.

C. Gorges, K. Öztürk, and R. Liebich, “Impact detection using a machine learning approach and experimental road roughness classification,” Mechanical Systems and Signal Processing, vol. 117, pp. 738–756, Feb. 2019, doi: 10.1016/j.ymssp.2018.07.043.

I. H. Sarker, “Machine Learning: Algorithms, Real-World Applications and Research Directions,” SN Comput Sci, vol. 2, no. 3, p. 160, 2021, doi: 10.1007/s42979-021-00592-x.

C. I. Agustyaningrum, M. Haris, R. Aryanti, and T. Misriati, “Online Shopper Intention Analysis Using Conventional Machine Learning and Deep Neural Network Classification Algorithm,” Jurnal Penelitian Pos dan Informatika, no. 1, p. 12, 2021.

L. J. Muhammad, E. A. Algehyne, S. S. Usman, A. Ahmad, C. Chakraborty, and I. A. Mohammed, “Supervised Machine Learning Models for Prediction of COVID-19 Infection using Epidemiology Dataset,” SN COMPUT. SCI., vol. 2, no. 1, p. 11, Nov. 2020, doi: 10.1007/s42979-020-00394-7.

A. M. Rahmani et al., “Machine Learning (ML) in Medicine: Review, Applications, and Challenges,” Mathematics, vol. 9, no. 22, p. 2970, Nov. 2021, doi: 10.3390/math9222970.

P. A. de L. Medeiros et al., “Efficient machine learning approach for volunteer eye-blink detection in real-time using webcam,” Expert Systems with Applications, vol. 188, p. 116073, Feb. 2022, doi: 10.1016/j.eswa.2021.116073.

B. R. Ibrahim et al., “Embedded System for Eye Blink Detection Using Machine Learning Technique,” in 2021 1st Babylon International Conference on Information Technology and Science (BICITS), Apr. 2021, pp. 58–62. doi: 10.1109/BICITS51482.2021.9509908.

C. Lugaresi et al., “MediaPipe: A Framework for Building Perception Pipelines.” arXiv, Jun. 14, 2019. doi: 10.48550/arXiv.1906.08172.

I. H. Sarker, M. H. Furhad, and R. Nowrozy, “AI-Driven Cybersecurity: An Overview, Security Intelligence Modeling and Research Directions,” SN COMPUT. SCI., vol. 2, no. 3, p. 173, Mar. 2021, doi: 10.1007/s42979-021-00557-0.

A. Ahmed, J. Guo, F. Ali, F. Deeba, and A. Ahmed, “LBPH based improved face recognition at low resolution,” in 2018 International Conference on Artificial Intelligence and Big Data (ICAIBD), Chengdu, China: IEEE, May 2018, pp. 144–147. doi: 10.1109/ICAIBD.2018.8396183.

“Deep Learning vs. Machine Learning: Beginner’s Guide,” Coursera, Mar. 23, 2023. https://www.coursera.org/articles/ai-vs-deep-learning-vs-machine-learning-beginners-guide (accessed Apr. 24, 2023).

G. Güney et al., “Video-Based Hand Movement Analysis of Parkinson Patients before and after Medication Using High-Frame-Rate Videos and MediaPipe,” Sensors, vol. 22, no. 20, Art. no. 20, Jan. 2022, doi: 10.3390/s22207992.

M. Knapik and B. Cyganek, “Fast eyes detection in thermal images,” Multimed Tools Appl, vol. 80, no. 3, pp. 3601–3621, Jan. 2021, doi: 10.1007/s11042-020-09403-6.

“Confusion Matrix - an overview | ScienceDirect Topics.” https://www.sciencedirect.com/topics/engineering/confusion-matrix (accessed Jul. 04, 2022).

M. Rasyid, Z. Zainuddin, and A. Andani, “Early Detection of Health Kindergarten Student at School Using Image Processing Technology,” in Proceedings of the 1st International Conference on Science and Technology, ICOST 2019, 2-3 May, Makassar, Indonesia, Makassar, Indonesia: EAI, 2019. doi: 10.4108/eai.2-5-2019.2284609.

“Confusion Matrix - an overview | ScienceDirect Topics.” Accessed: Jul. 04, 2022. [Online]. Available: https://www.sciencedirect.com/topics/engineering/confusion-matrix

M. F. Rasyid, A. A. M. Suradi, A. Arifin, M. Rizal, and M. Mushaf, “Utilization of Telegram application As an Information Media Face Mask Detection Result,” Sistemasi: Jurnal Sistem Informasi, vol. 12, no. 1, Art. no. 1, Jan. 2023, doi: 10.32520/stmsi.v12i1.2264.

Moh. A. Hasan, Y. Riyanto, and D. Riana, “Grape leaf image disease classification using CNN-VGG16 model,” J. Teknol. dan Sist. Komput, vol. 9, no. 4, pp. 218–223, Oct. 2021, doi: 10.14710/jtsiskom.2021.14013.

J. Xu, Y. Zhang, and D. Miao, “Three-way confusion matrix for classification: A measure driven view,” Information Sciences, vol. 507, pp. 772–794, Jan. 2020, doi: 10.1016/j.ins.2019.06.064.

“Automatic Detection of Tomato Leaf Deficiency using Soft Computing Technique,” IJEAT, vol. 9, no. 2, pp. 5406–5410, Dec. 2019, doi: 10.35940/ijeat.A1045.129219.

F. Utaminingrum, A. D. Purwanto, M. R. R. Masruri, K. Ogata, and I. K. Somawirata, “Eye Movement and Blink Detection for Selecting Menu On-Screen Display Using Probability Analysis Based on Facial Landmark.” ICIC International 学会, 2021. doi: 10.24507/ijicic.17.04.1287.

D. A. Navastara, W. Y. M. Putra, and C. Fatichah, “Drowsiness Detection Based on Facial Landmark and Uniform Local Binary Pattern,” J. Phys.: Conf. Ser., vol. 1529, no. 5, p. 052015, May 2020, doi: 10.1088/1742-6596/1529/5/052015.

A. Miyaji, Y. Kimata, T. Matsui, M. Fujimoto, and K. Yasumoto, “Analysis and Visualization of Relationship between Stress and Care Activities toward Reduction in Caregiver Workload,” Sensors and Materials, vol. 34, no. 8, p. 2929, Aug. 2022, doi: 10.18494/SAM3972.