Improved Micro-expression Recognition: An Apex Frame-Based Approach Feature Tracking and KLT
DOI:
https://doi.org/10.52436/1.jutif.2025.6.2.4235Keywords:
DRMF, emotion classification, feature tracking, KLT, micro-expressionAbstract
This research develops a real-time facial micro-expression recognition system, focusing on analyzing the onset and apex phases of micro-expression on the Spontaneous Activity and Micro-Movements (SAMM) dataset. Micro- expressions are very brief (0.04 - 0.2 seconds) facial muscle movements that often occur when a person is trying to hide emotions. The developed system aims to improve computation time efficiency and micro-expression recognition accuracy by optimizing feature extraction techniques and selecting more specific facial areas, including facial components such as eyebrows, eyes, and mouth. This research successfully improved the computation time efficiency by 51.96%, almost half the time required by the previous method. In addition, this study shows an increase in efficiency compared to previous studies, with an increase of 34.3% for SVM with Manual Sampling technique and 32.6% for MLP-Backpropagation. In the Random Sampling technique, SVM efficiency increased by 6.1%, but MLP-Backpropagation accuracy decreased by 4.8%. This method achieved 77.9% accuracy for MLP- Backpropagation, which is higher than the previous method. This research contributes to accelerating micro- expression recognition systems and improving accuracy, which opens opportunities for real-time emotion analysis applications such as lie detection or human behavior monitoring in a broader context.
Downloads
References
X. Zeng et al., “Affection of facial artifacts caused by micro-expressions on electroencephalography signals,” Front Neurosci, vol. 16, p. 1048199, Nov. 2022, doi: https://doi.org/10.3389/fnins.2022.1048199.
P. Ekman and W. V. Friesen, “Constants across cultures in the face and emotion,” Journal of Personality and Social Psychology, vol. 17, no. 2, pp. 124–129, 1971, doi: 10.1037/h0030377.
P. Ekman, “Lie catching and microexpressions,” The philosophy of deception, vol. 1, no. 2, p. 5, 2009.
X. Ben et al., “Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–1, 2021, doi: 10.1109/TPAMI.2021.3067464.
B. Allaert, I. M. Bilasco, and C. Djeraba, “Micro and Macro Facial Expression Recognition Using Advanced Local Motion Patterns,” IEEE Transactions on Affective Computing, vol. 13, no. 1, pp. 147–158, Jan. 2022, doi: 10.1109/TAFFC.2019.2949559.
N. L. Yee, M. A. Zulkifley, A. H. Saputro, and S. R. Abdani, “Apex Frame Spotting Using Attention Networks for Micro-Expression Recognition System,” Computers, Materials and Continua, vol. 73, no. 3, pp. 5331–5348, 2022, doi: 10.32604/cmc.2022.028801.
K. S. Widjaja, C. C. Alamo, Anderies, and A. Chowanda, “Exploring the Accuracy of Artificial Intelligence in Detecting Lies Through Micro-expression Analysis,” in 2023 6th International Conference on Information and Communications Technology (ICOIACT), Nov. 2023, pp. 218–
doi: 10.1109/ICOIACT59844.2023.10455941.
H.-X. Xie, L. Lo, H.-H. Shuai, and W.-H. Cheng, “An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge,” IEEE Transactions on Affective Computing, vol. 14, no. 3, pp. 1857–1875, Jul. 2023, doi: 10.1109/TAFFC.2022.3143100.
F. Zhang and L. Chai, “A review of research on micro-expression recognition algorithms based on deep learning,” Neural Comput & Applic, vol. 36, no. 29, pp. 17787–17828, Oct. 2024, doi: 10.1007/s00521-024-10262-7.
J. He, L. Peng, B. Sun, L. Yu, and M. Guo, “Dual Multi-Task Network with Bridge-Temporal- Attention for Student Emotion Recognition via Classroom Video,” in 2021 International Joint Conference on Neural Networks (IJCNN), Jul. 2021, pp. 1–8. doi: 10.1109/IJCNN52387.2021.9533471.
H. Zhang, L. Yin, and H. Zhang, “A review of micro-expression spotting: methods and challenges,” Multimedia Systems, vol. 29, no. 4, pp. 1897–1915, Aug. 2023, doi: 10.1007/s00530-023-01076-z.
G. Zhao, X. Li, Y. Li, and M. Pietikäinen, “Facial Micro-Expressions: An Overview,” Proceedings of the IEEE, vol. 111, no. 10, pp. 1215–1235, Oct. 2023, doi: 10.1109/JPROC.2023.3275192.
P. Choirina, U. D. Rosiani, and I. M. Fitriani, “Pelacakan Titik Fitur Dengan Kanade-Lucas- Tomasi (KLT) Untuk Pengenalan Ekspresi Mikro Wajah Pada Dataset SAMM,” G-Tech: Jurnal Teknologi Terapan, vol. 8, no. 1, pp. 330–339, 2024, doi: 10.33379/gtech.v8i1.3755.
R. Li et al., “Micro-Expression Recognition by Motion Feature Extraction based on Pre- training,” Jul. 10, 2024, arXiv: arXiv:2407.07345. doi: 10.48550/arXiv.2407.07345.
P. Ekman and W. V. Friesen, “Nonverbal Leakage and Clues to Deception†,” Psychiatry, vol. 32, no. 1, Art. no. 1, Feb. 1969, doi: 10.1080/00332747.1969.11023575.
S.-T. Liong, J. See, K. Wong, and R. C.-W. Phan, “Less is more: Micro-expression recognition from video using apex frame,” Signal Processing: Image Communication, vol. 62, pp. 82–92, Mar. 2018, doi: 10.1016/j.image.2017.11.006.
E. Friesen and P. Ekman, “Facial action coding system: a technique for the measurement of facial movement,” Palo Alto, vol. 3, no. 2, p. 5, 1978, doi: https://doi.org/10.1037/t27734-000.
I. V. Vitale, “Facial Action Coding System Applied to Criminal Investigations: The Analysis of a Homicide Case in Southern Italy,” International Annals of Criminology, vol. 59, no. 1, pp. 11– 22, May 2021, doi: 10.1017/cri.2021.7.
K. Lekdioui, Y. Ruichek, R. Messoussi, Y. Chaabi, and R. Touahni, “Facial expression recognition using face-regions,” in 2017 International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), May 2017, pp. 1–6. doi: 10.1109/ATSIP.2017.8075517.
U. D. Rosiani, P. Choirina, S. Sumpeno, and M. H. Purnomo, “Menuju Pengenalan Ekspresi Mikro: Pendeteksian Komponen Wajah Menggunakan Discriminative Response Map Fitting,” Jurnal Nasional Teknik Elektro dan Teknologi Informasi (JNTETI), vol. 7, no. 2, Art. no. 2, 2018, doi: 10.22146/jnteti.v7i2.424.
R. A. Pahlevi and B. Setiaji, “Analysis of Application Haar Cascade Classifier and Local Binary Pattern Histogram Algorithm in Recognizing Faces With Real-Time Grayscale Images Using Opencv,” Jurnal Teknik Informatika (Jutif), vol. 4, no. 1, pp. 179–186, 2023, doi: https://doi.org/10.52436/1.jutif.2023.4.1.491.
H. Rahman, M. U. Ahmed, and S. Begum, “Non-Contact Physiological Parameters Extraction Using Facial Video Considering Illumination, Motion, Movement and Vibration,” IEEE Transactions on Biomedical Engineering, vol. 67, no. 1, pp. 88–98, Jan. 2020, doi: 10.1109/TBME.2019.2908349.
K. Lee, D. K. Han, and H. Ko, “Video Analytic Based Health Monitoring for Driver in Moving Vehicle by Extracting Effective Heart Rate Inducing Features,” Journal of Advanced Transportation, vol. 2018, no. 1, p. 8513487, 2018, doi: 10.1155/2018/8513487.
P. Choirina, U. D. Rosiani, and I. M. Fitriani, “Pengenalan Ekspresi Mikro Wajah Berdasarkan Point Feature Tracking Menggunakan Fase Apex Pada Database Ekspresi Mikro,” Edu Komputika Journal, vol. 9, no. 1, Art. no. 1, Jun. 2022, doi: 10.15294/edukomputika.v9i1.56600.
A. Asthana, S. Zafeiriou, S. Cheng, and M. Pantic, “Robust Discriminative Response Map Fitting with Constrained Local Models,” in 2013 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2013, pp. 3444–3451. doi: 10.1109/CVPR.2013.442.
R. Miao, J. Qian, Y. Song, R. Ying, and P. Liu, “UniVIO: Unified Direct and Feature-Based Underwater Stereo Visual-Inertial Odometry,” IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 1–14, 2022, doi: 10.1109/TIM.2021.3136259.
R. A. Asmara et al., “Study of DRMF and ASM facial landmark point for micro expression recognition using KLT tracking point feature,” J. Phys.: Conf. Ser., vol. 1402, p. 077039, Dec. 2019, doi: 10.1088/1742-6596/1402/7/077039.
C. He, “Face tracking using Kanade-Lucas-Tomasi algorithm,” in Fifth International Conference on Computer Information Science and Artificial Intelligence (CISAI 2022), Y. Zhong, Ed., SPIE, 2023, p. 125663P. doi: 10.1117/12.2667789.
J.-C. Piao and S.-D. Kim, “Adaptive Monocular Visual–Inertial SLAM for Real-Time Augmented Reality Applications in Mobile Devices,” Sensors, vol. 17, no. 11, Art. no. 11, Nov. 2017, doi: 10.3390/s17112567.
J. Devasagayam, R. Bosma, and C. M. Collier, “A velocity program using the Kanade–Lucas– Tomasi feature-tracking algorithm with demonstration for pressure and electroosmosis conditions,” ELECTROPHORESIS, vol. 43, no. 7–8, pp. 865–878, 2022, doi: 10.1002/elps.202100177.
Z. Jun, “The Development and Application of Support Vector Machine,” J. Phys.: Conf. Ser., vol. 1748, no. 5, p. 052006, Jan. 2021, doi: 10.1088/1742-6596/1748/5/052006.
J. Zhang, C. Li, Y. Yin, J. Zhang, and M. Grzegorzek, “Applications of artificial neural networks in microorganism image analysis: a comprehensive review from conventional multilayer perceptron to popular convolutional neural network and potential visual transformer,” Artif Intell Rev, vol. 56, no. 2, pp. 1013–1070, Feb. 2023, doi: 10.1007/s10462-022-10192-7.
G. Garg, D. Kumar, ArvinderPal, Y. Sonker, and R. Garg, “A Hybrid MLP-SVM Model for Classification using Spatial-Spectral Features on Hyper-Spectral Images,” Jan. 01, 2021, arXiv: arXiv:2101.00214. doi: 10.48550/arXiv.2101.00214.
A. K. Davison, C. Lansley, N. Costen, K. Tan, and M. H. Yap, “SAMM: A Spontaneous Micro- Facial Movement Dataset,” IEEE Transactions on Affective Computing, vol. 9, no. 1, pp. 116– 129, Jan. 2018, doi: 10.1109/TAFFC.2016.2573832.
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Indah Martha Fitriani, Priska Choirina, Ulla Delfana Rosiani, Muhammad Nabil Mufti, Firmanda Ahmadani Arsistawa
This work is licensed under a Creative Commons Attribution 4.0 International License.
