Incremental CNN-k-NN Hybrid Facial Recognition for Helmeted Facial Recognition in IoT-Enabled Smart Parking: A Case Study at Universitas Mataram
DOI:
https://doi.org/10.52436/1.jutif.2025.6.6.5447Keywords:
facial recognition, incremental learning, hybrid CNN-k-NN classification, smart parking, university IoTAbstract
Helmeted rider identification challenges traditional facial recognition, especially in Indonesian campuses like UNRAM, where motorbike use is prevalent and theft risks are high. This study develops a hybrid CNN-k-NN system for secure parking access. The dataset contains 2,800 augmented images (Haar Cascade crop, 224x224 grayscale), with features extracted via VGG16/ResNet and classified using k-NN (k=1, Euclidean/Cosine). The system achieves 95.62% accuracy, with precision, recall, and F1 scores of 0.96. Incremental retraining reduces processing time to under 1 second, compared to 30 minutes for full retraining. The use of cosine similarity improves accuracy slightly over Euclidean distance. This solution enhances IoT-based smart campuses by enabling efficient, real-time identification and reducing theft by improving access control. It is adaptable to low-resource environments, supporting scalable deployments in smart parking and campus security systems.
Downloads
References
Y.-C. Huang, D. A. B. Rahardjo, R.-H. Shiue, and H. H. Chen, “Masked face recognition using domain adaptation,” Pattern Recognition, vol. 153, p. 110574, Sept. 2024, doi: 10.1016/j.patcog.2024.110574.
Y. Song and S. Liu, “A Deep Hierarchical Feature Sparse Framework for Occluded Person Re-Identification,” Jan. 15, 2024, arXiv: arXiv:2401.07469. doi: 10.48550/arXiv.2401.07469.
S. Tyagi, V. Gupta, and V. Mehndiratta, “Building Smart Campuses: Integrating AI in Higher Education,” in Recent Trends in Artificial Intelligence Towards a Smart World, R. Arya, S. C. Sharma, A. K. Verma, and B. Iyer, Eds., in Frontiers of Artificial Intelligence, Ethics and Multidisciplinary Applications. , Singapore: Springer Nature Singapore, 2024, pp. 399–431. doi: 10.1007/978-981-97-6790-8_15.
T. Sutjarittham, “Modelling and Optimisation of Resource Usage in an IoT Enabled Smart Campus,” 2021, arXiv. doi: 10.48550/ARXIV.2111.04085.
D. Rico-Bautista, Y. Medina-Cardenas, L. A. Coronel-Rojas, F. Cuesta-Quintero, G. Maestre-Gongora, and C. D. Guerrero, “Smart University: Key Factors for an Artificial Intelligence Adoption Model,” in Advances and Applications in Computer Science, Electronics and Industrial Engineering, vol. 1307, M. V. García, F. Fernández-Peña, and C. Gordón-Gallegos, Eds., in Advances in Intelligent Systems and Computing, vol. 1307. , Singapore: Springer Singapore, 2021, pp. 153–166. doi: 10.1007/978-981-33-4565-2_10.
M. O. Oloyede, G. P. Hancke, and H. C. Myburgh, “A review on face recognition systems: recent approaches and challenges,” Multimed Tools Appl, vol. 79, no. 37–38, pp. 27891–27922, Oct. 2020, doi: 10.1007/s11042-020-09261-2.
M. Xue, X. Duan, W. Liu, and Y. Ren, “A semantic facial expression intensity descriptor based on information granules,” Information Sciences, vol. 528, pp. 113–132, Aug. 2020, doi: 10.1016/j.ins.2020.04.012.
M. Sajjad et al., “A comprehensive survey on deep facial expression recognition: challenges, applications, and future guidelines,” Alexandria Engineering Journal, vol. 68, pp. 817–840, Apr. 2023, doi: 10.1016/j.aej.2023.01.017.
E. Barcic, P. Grd, and I. Tomicic, “Convolutional Neural Networks for Face Recognition: A Systematic Literature Review,” July 12, 2023, In Review. doi: 10.21203/rs.3.rs-3145839/v1.
A. I. Awad, A. Babu, E. Barka, and K. Shuaib, “AI-powered biometrics for Internet of Things security: A review and future vision,” Journal of Information Security and Applications, vol. 82, p. 103748, May 2024, doi: 10.1016/j.jisa.2024.103748.
S. M. Arman, T. Yang, S. Shahed, A. A. Mazroa, A. Attiah, and L. Mohaisen, “A Comprehensive Survey for Privacy-Preserving Biometrics: Recent Approaches, Challenges, and Future Directions,” CMC, vol. 78, no. 2, pp. 2087–2110, 2024, doi: 10.32604/cmc.2024.047870.
H. Deng, Z. Feng, G. Qian, X. Lv, H. Li, and G. Li, “MFCosface: A Masked-Face Recognition Algorithm Based on Large Margin Cosine Loss,” Applied Sciences, vol. 11, no. 16, p. 7310, Aug. 2021, doi: 10.3390/app11167310.
F. Schroff, D. Kalenichenko, and J. Philbin, “FaceNet: A Unified Embedding for Face Recognition and Clustering,” 2015, doi: 10.48550/ARXIV.1503.03832.
Q. Cao, L. Shen, W. Xie, O. M. Parkhi, and A. Zisserman, “VGGFace2: A Dataset for Recognising Faces across Pose and Age,” in 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi’an: IEEE, May 2018, pp. 67–74. doi: 10.1109/FG.2018.00020.
P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. Fisherfaces: recognition using class specific linear projection,” IEEE Trans. Pattern Anal. Machine Intell., vol. 19, no. 7, pp. 711–720, July 1997, doi: 10.1109/34.598228.
S.-K. Kim, Y. J. Park, K.-A. Toh, and S. Lee, “SVM-based feature extraction for face recognition,” Pattern Recognition, vol. 43, no. 8, pp. 2871–2881, Aug. 2010, doi: 10.1016/j.patcog.2010.03.008.
O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep Face Recognition,” in Procedings of the British Machine Vision Conference 2015, Swansea: British Machine Vision Association, 2015, p. 41.1-41.12. doi: 10.5244/C.29.41.
S. Zheng, R. W. O. Rahmat, F. Khalid, and N. A. Nasharuddin, “3D texture-based face recognition system using fine-tuned deep residual networks,” PeerJ Computer Science, vol. 5, p. e236, Dec. 2019, doi: 10.7717/peerj-cs.236.
A. J. O’Toole and C. D. Castillo, “Face Recognition by Humans and Machines: Three Fundamental Advances from Deep Learning,” Annu. Rev. Vis. Sci., vol. 7, no. 1, pp. 543–570, Sept. 2021, doi: 10.1146/annurev-vision-093019-111701.
C. Cortes and V. Vapnik, “Support-vector networks,” Mach Learn, vol. 20, no. 3, pp. 273–297, Sept. 1995, doi: 10.1007/BF00994018.
L. Breiman, “Random Forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001, doi: 10.1023/A:1010933404324.
Popescu, Marius-Constantin and Balas, Valentina E. and Perescu-Popescu, Liliana and Mastorakis, Nikos, “Multilayer perceptron and neural networks,” World Scientific and Engineering Academy and Society (WSEAS), vol. 8, no. 7, pp. 579–588, 2009.
X. Tang, L. Zhang, and X. Ding, “SAR image despeckling with a multilayer perceptron neural network,” International Journal of Digital Earth, vol. 12, no. 3, pp. 354–374, Mar. 2019, doi: 10.1080/17538947.2018.1447032.
K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” Apr. 10, 2015, arXiv: arXiv:1409.1556. doi: 10.48550/arXiv.1409.1556.
J. Han and M. Kamber, Data mining: concepts and techniques, 3rd ed. Burlington, MA: Elsevier, 2012.
J. Brownlee, Machine learning mastery with Python: understand your data, create accurate models and work projects end-to-end, Edition: v1.20. Australia: Jason Brownlee, 2021.
Aurélien Géron, Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow. Beijing Boston Farnham Sebastopol Tokyo: O’REILLY, 2019.
S. Zheng, R. W. O. Rahmat, F. Khalid, and N. A. Nasharuddin, “3D texture-based face recognition system using fine-tuned deep residual networks,” PeerJ Computer Science, vol. 5, p. e236, Dec. 2019, doi: 10.7717/peerj-cs.236.
H. Deng, Z. Feng, G. Qian, X. Lv, H. Li, and G. Li, “MFCosface: A Masked-Face Recognition Algorithm Based on Large Margin Cosine Loss,” Applied Sciences, vol. 11, no. 16, p. 7310, Aug. 2021, doi: 10.3390/app11167310.
X. Xiao, Y. Zhang, T. Liu, and J. Duan, “Deep Learning for Event-Driven Stock Prediction,” in The Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 2015), Argentina, July 2015.
G. R. Nahass et al., “FaceFinder: A machine learning tool for identification of facial images from heterogenous datasets,” AJO International, vol. 1, no. 4, p. 100083, Dec. 2024, doi: 10.1016/j.ajoint.2024.100083.
A. Ettiyappan, H. K. Palani, K. Duraipandian, and N. Selvaganapathy, “High-accuracy face recognition with the MRYOLO-SCO algorithm: combining residual networks and YOLOv2 for improved precision,” SIViP, vol. 19, no. 7, p. 560, July 2025, doi: 10.1007/s11760-025-04053-3.
P. Fränti and R. Mariescu-Istodor, “Soft precision and recall,” Pattern Recognition Letters, vol. 167, pp. 115–121, Mar. 2023, doi: 10.1016/j.patrec.2023.02.005.
A. Tasnim, Md. Saiduzzaman, M. A. Rahman, J. Akhter, and A. S. Md. M. Rahaman, “Performance Evaluation of Multiple Classifiers for Predicting Fake News,” JCC, vol. 10, no. 09, pp. 1–21, 2022, doi: 10.4236/jcc.2022.109001.
M. Eman, T. M. Mahmoud, M. M. Ibrahim, and T. Abd El-Hafeez, “Innovative Hybrid Approach for Masked Face Recognition Using Pretrained Mask Detection and Segmentation, Robust PCA, and KNN Classifier,” Sensors, vol. 23, no. 15, p. 6727, July 2023, doi: 10.3390/s23156727.
H. He, J. Liang, Z. Hou, H. Liu, and X. Zhou, “Occlusion recovery face recognition based on information reconstruction,” Machine Vision and Applications, vol. 34, no. 5, p. 74, Sept. 2023, doi: 10.1007/s00138-023-01423-0.
Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, “DeepFace: Closing the Gap to Human-Level Performance in Face Verification,” in 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA: IEEE, June 2014, pp. 1701–1708. doi: 10.1109/CVPR.2014.220.
E. Lopez-Lopez, X. M. Pardo, and C. V. Regueiro, “Incremental Learning from Low-labelled Stream Data in Open-Set Video Face Recognition,” Pattern Recognition, vol. 131, p. 108885, Nov. 2022, doi: 10.1016/j.patcog.2022.108885.
E. L. Piza, B. C. Welsh, D. P. Farrington, and A. L. Thomas, “CCTV surveillance for crime prevention: A 40‐year systematic review with meta‐analysis,” Criminology & Public Policy, vol. 18, no. 1, pp. 135–159, Feb. 2019, doi: 10.1111/1745-9133.12419.
Q. T. Van et al., “Intelligent Parking System Using Automated License Plate Recognition and Face Verification,” in Proceedings of International Conference on Computing and Communication Networks, vol. 394, A. K. Bashir, G. Fortino, A. Khanna, and D. Gupta, Eds., in Lecture Notes in Networks and Systems, vol. 394. , Singapore: Springer Nature Singapore, 2022, pp. 219–227. doi: 10.1007/978-981-19-0604-6_20.
T. Thaher, M. Mafarja, M. Saffarini, A. H. H. M. Mohamed, and A. A. El-Saleh, “A Comprehensive Review of Face Detection Techniques for Occluded Faces: Methods, Datasets, and Open Challenges,” CMES, vol. 143, no. 3, pp. 2615–2673, 2025, doi: 10.32604/cmes.2025.064857.
M. Mahmoud, M. S. Kasem, and H.-S. Kang, “A Comprehensive Survey of Masked Faces: Recognition, Detection, and Unmasking,” Applied Sciences, vol. 14, no. 19, p. 8781, Sept. 2024, doi: 10.3390/app14198781.
K. Marwa and O. Kais, “Classifiers Combination for Efficient Masked Face Recognition,” IJACSA, vol. 13, no. 9, 2022, doi: 10.14569/IJACSA.2022.01309120.
M. Hu, Y. Wu, Y. Yang, J. Fan, and B. Jing, “DAGL-Faster: Domain adaptive faster r-cnn for vehicle object detection in rainy and foggy weather conditions,” Displays, vol. 79, p. 102484, Sept. 2023, doi: 10.1016/j.displa.2023.102484.
B. Huang et al., “When Face Recognition Meets Occlusion: A New Benchmark,” Mar. 04, 2021, arXiv: arXiv:2103.02805. doi: 10.48550/arXiv.2103.02805.
H. Qian, P. Zhang, S. Ji, S. Cao, and Y. Xu, “Improving Representation Consistency with Pairwise Loss for Masked Face Recognition,” in 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Montreal, BC, Canada: IEEE, Oct. 2021, pp. 1462–1467. doi: 10.1109/ICCVW54120.2021.00169.
Z. Zhang, X. Ji, X. Cui, and J. Ma, “A Survey on Occluded Face recognition,” in 2020 The 9th International Conference on Networks, Communication and Computing, Tokyo Japan: ACM, Dec. 2020, pp. 40–49. doi: 10.1145/3447654.3447661.
S. Malakar, W. Chiracharit, and K. Chamnongthai, “Masked Face Recognition With Generated Occluded Part Using Image Augmentation and CNN Maintaining Face Identity,” IEEE Access, vol. 12, pp. 126356–126375, 2024, doi: 10.1109/ACCESS.2024.3446652.
T. Thaher, M. Mafarja, M. Saffarini, A. H. H. M. Mohamed, and A. A. El-Saleh, “A Comprehensive Review of Face Detection Techniques for Occluded Faces: Methods, Datasets, and Open Challenges,” CMES, vol. 143, no. 3, pp. 2615–2673, 2025, doi: 10.32604/cmes.2025.064857.
Z. Deng, H. Chiang, L. Kang, and H. Li, “A lightweight deep learning model for real‐time face recognition,” IET Image Processing, vol. 17, no. 13, pp. 3869–3883, Nov. 2023, doi: 10.1049/ipr2.12903.
A. Sufian, A. Ghosh, D. Barman, M. Leo, C. Distante, and B. Li, “FewFaceNet: A Lightweight Few-Shot Learning-based Incremental Face Authentication for Edge Cameras,” in 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Paris, France: IEEE, Oct. 2023, pp. 2010–2019. doi: 10.1109/ICCVW60793.2023.00216.
F. Izourane, S. Ardchir, S. Ounacer, and M. Azzouazi, “Smart Campus Based on AI and IoT in the Era of Industry 5.0: Challenges and Opportunities,” in Industry 5.0 and Emerging Technologies, vol. 565, A. Chakir, R. Bansal, and M. Azzouazi, Eds., in Studies in Systems, Decision and Control, vol. 565. , Cham: Springer Nature Switzerland, 2024, pp. 39–57. doi: 10.1007/978-3-031-70996-8_3.
T. Domínguez-Bolaño, V. Barral, C. J. Escudero, and J. A. García-Naya, “An IoT system for a smart campus: Challenges and solutions illustrated over several real-world use cases,” 2024, doi: 10.48550/ARXIV.2403.15395.
N. El Fadel, “Facial Recognition Algorithms: A Systematic Literature Review,” J. Imaging, vol. 11, no. 2, p. 58, Feb. 2025, doi: 10.3390/jimaging11020058.
J. Johnson, M. Douze, and H. Jégou, “Billion-scale similarity search with GPUs,” 2017, arXiv. doi: 10.48550/ARXIV.1702.08734.
M. Douze et al., “The Faiss library,” 2024, arXiv. doi: 10.48550/ARXIV.2401.08281.
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Ida Bagus Ketut Widiartha, Ario Yudo Husodo, Tran Thi Thanh Thuy, Santi Ika Murpratiwi
This work is licensed under a Creative Commons Attribution 4.0 International License.
