CLOTHING RECOMMENDATION AND FACE SWAP MODEL BASED ON VGG16, AUTOENCODER, AND FACIAL LANDMARK POINTS

Imada Ramadhanti; Agi Prasetiadi; Iqsyahiro Kresna A

doi:10.52436/1.jutif.2024.5.1.1016

Imada Ramadhanti Informatics, Faculty of Informatics, Institut Teknologi Telkom Purwokerto, Indonesia
Agi Prasetiadi Informatics, Faculty of Informatics, Institut Teknologi Telkom Purwokerto, Indonesia
Iqsyahiro Kresna A Informatics, Faculty of Informatics, Institut Teknologi Telkom Purwokerto, Indonesia

DOI: https://doi.org/10.52436/1.jutif.2024.5.1.1016

Keywords: Autoencoder, Clothes, Dense, Landmark, VGG16

Abstract

The selection of clothes in e-commerce sometimes contains doubts about the clothes that consumers choose because the clothes are not yet known to suit the consumer's body. So this research provides a solution through a clothing recommendation model according to the size and concept of clothing. Furthermore, there is a face exchange model whose job is to exchange faces between the consumer's face and the face on the recommended clothing. The dataset used in the classification model is clothing that is put into 8 classes with variations in size, clothing concept, and veiled or without headscarves, while making the autoencoder model requires source and target face datasets of 3,000 faces each. The method used to make clothing model recommendations is VGG16 and the face exchange model uses the autoencoder and facial landmark points methods. The results of the classification model with 2 different architectures obtain an accuracy of 97.01% and 94.49% respectively. Then the results of the autoencoder models for the 12 models produced the lowest loss values with autoencoder I of 0.00012951 and in autoencoder II of 8.01e-05. The face landmark point method is used if the autoencoder method does not produce a good face swap.

Downloads

Download data is not yet available.

References

F. Lianto, R. Trisno, and D. Husin, “A portable architecture with an interior fashion concept,” Int. J. Eng. Res. Technol., vol. 12, no. 12, pp. 2126–2132, 2019.

F. Alfiah, R. Tarmizi, and A. A. Junidar, “Perancangan Sistem E--Commerce Untuk Penjualan Pakaian Pada Toko a&S,” Innov. Creat. Inf. Technol., vol. 6, no. 1, pp. 70–81, 2020.

B. Romano, S. Sands, and J. I. Pallant, “Augmented reality and the customer journey: an exploratory study,” Australas. Mark. J., vol. 29, no. 4, pp. 354–363, 2021.

R. Velita, A. R. P. Barusman, and V. Saptarini, “Pengaruh e-Wom dan Review Produk pada Market Place Shopee Terhadap Keputusan Pembelian Pakaian Jadi Di Bandar Lampung,” VISIONIST, vol. 8, no. 1, 2019.

J. Lv and X. Liu, “The Impact of Information Overload of E-Commerce Platform on Consumer Return Intention: Considering the Moderating Role of Perceived Environmental Effectiveness,” Int. J. Environ. Res. Public Health, vol. 19, no. 13, p. 8060, 2022.

S. Tammina, “Transfer learning using vgg-16 with deep convolutional neural network for classifying images,” Int. J. Sci. Res. Publ., vol. 9, no. 10, pp. 143–150, 2019.

B. Peng, H. Fan, W. Wang, J. Dong, and S. Lyu, “A Unified Framework for High Fidelity Face Swap and Expression Reenactment,” IEEE Trans. Circuits Syst. Video Technol., p. 1, 2021, doi: 10.1109/TCSVT.2021.3106047.

C. Sadu and P. K. Das, “Swapping Face Images Based on Augmented Facial Landmarks and Its Detection,” in 2020 IEEE REGION 10 CONFERENCE (TENCON), 2020, pp. 456–461. doi: 10.1109/TENCON50793.2020.9293884.

A. S. Sinaga, A. S. Sitio, and S. Dewi, “Identification of Biometric Deepfakes using Feature Learning Deep Learning,” J. Tek. Inform., vol. 3, no. August, pp. 1125–1130, 2022.

Öztürk, “Stacked auto-encoder based tagging with deep features for content-based medical image retrieval,” Expert Syst. Appl., vol. 161, p. 113693, 2020.

M. A. Rizaldi and E. R. Kaburuan, “Implementasi OCR dengan Metode Autoencoder untuk verifikasi data KTP,” J. Komput. Terap., vol. 8, no. 2, pp. 307–315, 2022.

Z. Cheng, H. Sun, M. Takeuchi, and Jiro Katto, “Energy Compaction-Based Image Compression Using Convolutional AutoEncoder,” IEEE Trans. Multimed., vol. 22, no. 4, pp. 860–873, 2020, doi: doi: 10.1109/TMM.2019.2938345.

S. H. Hong, S. Ryu, J. Lim, and W. Y. Kim, “Molecular Generative Model Based on an Adversarially Regularized Autoencoder,” J. Chem. Inf. Model., vol. 60, no. 1, pp. 29–36, Jan. 2020, doi: 10.1021/acs.jcim.9b00694.

N. Li and F. Chang, “Video anomaly detection and localization via multivariate gaussian fully convolution adversarial autoencoder,” Neurocomputing, vol. 369, pp. 92–105, 2019, doi: https://doi.org/10.1016/j.neucom.2019.08.044.

K. Pawar and V. Z. Attar, “Assessment of autoencoder architectures for data representation,” in Deep Learning: Concepts and Architectures, Springer, 2019, pp. 101–132.

Ulla Delfana Rosiani, Rosa Andrie Asmara, and Nadhifatul Laeily, “Penerapan Facial Landmark Point Untuk Klasifikasi Jenis Kelamin Berdasarkan Citra Wajah,” J. Inform. Polinema, vol. 6, no. 1, pp. 55–60, 2020, doi: 10.33795/jip.v6i1.328.

Andre Hartoko Aji Putra Perdana, Susijanto Tri Rasmana, and Heri Pratikno, “Implementasi Sistem Deteksi Mata Kantuk Berdasarkan Facial Landmarks Detection Menggunakan Metode Regression Trees,” J. Technol. Informatics, vol. 1, no. 1, pp. 1–9, 2019, doi: 10.37802/joti.v1i1.1.

W. Hutamaputra and F. Utaminingrum, “Implementasi Facial Landmark dalam Pengenalan Wajah pada Sistem Pembayaran Elektronik,” Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 5, no. 5, pp. 2058–2064, 2021.

W. Wang, X. Chen, S. Zheng, and H. Li, “Fast head pose estimation via rotation-adaptive facial landmark detection for video edge computation,” IEEE Access, vol. 8, pp. 45023–45032, 2020.

A. Y. F. Rambe and L. D. Afri, “Analisis kemampuan pemecahan masalah matematis siswa dalam menyelesaikan soal materi barisan dan deret,” AXIOM J. Pendidik. Dan Mat., vol. 9, no. 2, pp. 175–187, 2020.