Multimodal Biometric Recognition Based on Fusion of Electrocardiogram and Fingerprint Using CNN, LSTM, CNN-LSTM, and DNN Models

Winda  Agustina; Dodon Turianto Nugrahadi; Mohammad Reza Faisal; Triando Hamonangan Saragih; Andi Farmadi; Irwan Budiman; Jumadi Mabe Parenreng; Muhammad  Alkaff

doi:10.52436/1.jutif.2025.6.4.5098

Authors

Winda Agustina Department of Computer Science, Lambung Mangkurat University, Banjarbaru, Indonesia
Dodon Turianto Nugrahadi Department of Computer Science, Lambung Mangkurat University, Banjarbaru, Indonesia
Mohammad Reza Faisal Department of Computer Science, Lambung Mangkurat University, Banjarbaru, Indonesia
Triando Hamonangan Saragih Department of Computer Science, Lambung Mangkurat University, Banjarbaru, Indonesia
Andi Farmadi Department of Computer Science, Lambung Mangkurat University, Banjarbaru, Indonesia
Irwan Budiman Department of Computer Science, Lambung Mangkurat University, Banjarbaru, Indonesia
Jumadi Mabe Parenreng Informatics and Computer Engineering Department, Universitas Negeri Makassar, Indonesia
Muhammad Alkaff Computer Science Department, Faculty of Computing and Information Technology, King Abdul Aziz University, Jeddah, Saudi Arabia

DOI:

https://doi.org/10.52436/1.jutif.2025.6.4.5098

Keywords:

Biometric Recognition, CNN, ECG, Feature Fusion, Fingerprint, Multimodal

Abstract

Biometric authentication offers a promising solution for enhancing the security of digital systems by leveraging individuals' unique physiological characteristics. This study proposes a multimodal authentication system using deep learning approaches to integrate fingerprint images and electrocardiogram (ECG) signals. The datasets employed include FVC2004 for fingerprint data and ECG-ID for ECG signals. Four deep learning architectures—Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), CNN-LSTM, and Deep Neural Network (DNN)—are evaluated to compare their effectiveness in recognizing individual identity based on fused multimodal features. Feature extraction techniques include grayscale conversion, binarization, edge detection, minutiae extraction for fingerprint images, and R-peak–based segmentation for ECG signals. The extracted features are combined using a feature-level fusion strategy to form a unified representation. Experimental results indicate that the CNN model achieves the highest classification accuracy at 96.25%, followed by LSTM and DNN at 93.75%, while CNN-LSTM performs the lowest at 11.25%. Minutiae-based features consistently yield superior results across different models, highlighting the importance of local feature descriptors in fingerprint-based identification tasks. This research advances biometric authentication by demonstrating the effectiveness of feature-level fusion and CNN architecture for accurate and robust identity recognition. The proposed system shows strong potential for secure and adaptive biometric authentication in modern digital applications.

Downloads

Download data is not yet available.

References

A.-R. Despa, "Overcoming Obstacles: A Closer Look at the Most Pressing Challenges in Cybersecurity," International Journal of Information Security and Cybercrime, p., 2023, doi: 10.19107/ijisc.2023.02.04.

Sushil Mahato, Ranjit Sah, and Sushil Sapkota, "Cybersecurity Challenges and Threats: The Risks in Digital World," International Journal of Advanced Research in Science, Communication and Technology, pp. 651–655, Nov. 2024, doi: 10.48175/IJARSCT-22497.

N. A. Mirjat, "AI and Machine Learning: Transforming the Landscape of Cybersecurity," Bulletin of Engineering Science and Technology, vol. 1, pp. 40–59, 2024, [Online]. Available: https://consensus.app/papers/ai-and-machine-learning-transforming-the-landscape-of-mirjat/449c39cd69905a149652e029a3b4e067/

S. Pandey, "Cybersecurity Trends and Challenges," INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT, vol. 07, no. 08, Aug. 2023, doi: 10.55041/ijsrem25323.

A. Dinesh, C. D. P. Reddy, G. Gopi, R. Jain, and T. Shankar, "A Durable Biometric Authentication Scheme via Blockchain," 2021 International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT), pp. 1–5, 2021, doi: 10.1109/ICAECT49130.2021.9392415.

A. Sindar, A. S. Sinaga, A. S. Sitio, and S. Dewi, "IDENTIFICATION OF BIOMETRIC DEEPFAKES USING FEATURE LEARNING DEEP LEARNING," vol. 3, no. 4, 2022, doi: 10.20884/1.jutif.2022.3.4.461.

A. Kailas and G. N. Keshava Murthy, "Deep learning based biometric authentication using electrocardiogram and iris," IAES International Journal of Artificial Intelligence, vol. 13, no. 1, pp. 1090–1103, Mar. 2024, doi: 10.11591/ijai.v13.i1.pp1090-1103.

R. Kumar, "Biometrics and Password Less Authentication: The Future of Digital Security," International Journal of Innovative Science and Research Technology (IJISRT), p., 2024, doi: 10.38124/ijisrt/ijisrt24jul541.

M. I. M. Yusop, N. H. Kamarudin, N. H. S. Suhaimi, and M. K. Hasan, "Advancing Passwordless Authentication: A Systematic Review of Methods, Challenges, and Future Directions for Secure User Identity," IEEE Access, vol. 13, pp. 13919–13943, 2025, doi: 10.1109/ACCESS.2025.3528960.

T. Oduguwa and A. Arabo, "Passwordless Authentication Using a Combination of Cryptography, Steganography, and Biometrics," Journal of Cybersecurity and Privacy, vol. 4, no. 2, pp. 278–297, Jun. 2024, doi: 10.3390/jcp4020014.

A. Jackson et al., "Biometric Authentication for the Mitigation of Human Risk on a Social Network," in Human Factors in Cybersecurity, AHFE International, 2024. doi: 10.54941/ahfe1004763.

N. D. Miranda, L. Novamizanti, and S. Rizal, “CONVOLUTIONAL NEURAL NETWORK PADA KLASIFIKASI SIDIK JARI MENGGUNAKAN RESNET-50,” Jurnal Teknik Informatika (Jutif), vol. 1, no. 2, pp. 61–68, Dec. 2020, doi: 10.20884/1.jutif.2020.1.2.18.

U. Sumalatha, K. K. Prakasha, S. Prabhu, and V. C. Nayak, "A Comprehensive Review of Unimodal and Multimodal Fingerprint Biometric Authentication Systems: Fusion, Attacks, and Template Protection," IEEE Access, vol. 12, pp. 64300–64334, 2024, doi: 10.1109/ACCESS.2024.3395417.

Diptadip Maiti and Madhuchhanda Basak, "Multimodal biometric integration: Trends and insights from the past quinquennial," World Journal of Advanced Research and Reviews, vol. 23, no. 3, pp. 1590–1605, Sep. 2024, doi: 10.30574/wjarr.2024.23.3.2741.

Lakshmi Narayana Gupta Koralla, "Biometric data and behavior analysis," World Journal of Advanced Research and Reviews, vol. 26, no. 1, pp. 339–350, Apr. 2025, doi: 10.30574/wjarr.2025.26.1.1084.

C. Yuan, S. Jiao, X. Sun, and Q. Wu, "MFFFLD: A Multimodal-Feature-Fusion-Based Fingerprint Liveness Detection," IEEE Trans Cogn Dev Syst, vol. 14, pp. 648–661, 2021, doi: 10.1109/TCDS.2021.3062624.

M. H. Safavipour, M. A. Doostari, and H. Sadjedi, "A hybrid approach to multimodal biometric recognition based on feature-level fusion of face, two irises, and both thumbprints," J Med Signals Sens, vol. 12, no. 3, pp. 177–191, Jul. 2022, doi: 10.4103/jmss.jmss_103_21.

Y. Wang, D. Shi, and W. Zhou, "Convolutional Neural Network Approach Based on Multimodal Biometric System with Fusion of Face and Finger Vein Features," Sensors, vol. 22, no. 16, Aug. 2022, doi: 10.3390/s22166039.

W. Yang, S. Wang, J. Hu, X. Tao, and Y. Li, "Feature extraction and learning approaches for cancellable biometrics: A survey," Feb. 01, 2024, John Wiley and Sons Inc. doi: 10.1049/cit2.12283.

H. Mehraj and A. Mir, "Robust Multimodal Biometric System Based on Feature Level Fusion of Optimiseddeepnet Features," Wirel Pers Commun, vol. 127, pp. 2461–2482, 2021, doi: 10.1007/s11277-021-09075-x.

A. Hussian, F. Murshed, M. N. Al-Andoli, and G. Aljafari, "A Hybrid Deep Learning Approach for Secure Biometric Authentication Using Fingerprint Data," Computers, p., 2025, doi: 10.3390/computers14050178.

H. Bendjenna, Y. Belhocine, A. Khemane, and A. Meraoumia, "Beyond Traditional Methods: Feature Fusion for Robust Biometric Identification," 2024 6th International Conference on Pattern Analysis and Intelligent Systems (PAIS), pp. 1–8, 2024, doi: 10.1109/PAIS62114.2024.10541249.

M. Kumar and D. Kumar, "An efficient gravitational search decision forest approach for fingerprint recognition."

D. L. Andreea-Monica, S. Moldovanu, and L. Moraru, "A Fingerprint Matching Algorithm Using the Combination of Edge Features and Convolution Neural Networks," Inventions, vol. 7, no. 2, Jun. 2022, doi: 10.3390/inventions7020039.

F. Taliningsih et al., "Biometric Verification Based on ECG Signal using 1 Dimensional Convolutional Neural Network," 2022 1st International Conference on Information System & Information Technology (ICISIT), pp. 204–209, 2022, doi: 10.1109/ICISIT54091.2022.9872891.

D. Nugrahadi, M. Faisal, R. Herteno, I. Budiman, F. Abadi, and I. Sutedja, "An Effective Preprocessing Data on Performance of Machine Learning for ECG-Based Personal Authentication," Proceedings of the 8th International Conference on Sustainable Information Engineering and Technology, p., 2023, doi: 10.1145/3626641.3626943.

M. Ramkumar, M. Alagarsamy, A. Balakumar, and S. Pradeep, "Ensemble classifier fostered detection of arrhythmia using ECG data," Med Biol Eng Comput, vol. 61, pp. 2453–2466, 2023, doi: 10.1007/s11517-023-02839-6.

M. Irhamsyah, M. Melinda, J. Alifa, J. Prayoga, and Y. Iskandar, "ECG Atrial Fibrillation Signal Classification Method Based on Discrete Wavelet Transform (DWT) and DenseNet-121," 2024 10th International Conference on Smart Computing and Communication (ICSCC), pp. 619–624, 2024, doi: 10.1109/ICSCC62041.2024.10690792.

M. Hirsi Mohamed, "Fingerprint Classification Using Deep Convolutional Neural Network," Journal of Electrical and Electronic Engineering, vol. 9, no. 5, p. 147, 2021, doi: 10.11648/j.jeee.20210905.11.

S. Khan et al., "A Deep Learning Framework for the Classification of ECG Signals," in 2022 International Conference on Engineering and Emerging Technologies (ICEET), 2022, pp. 1–5. doi: 10.1109/ICEET56468.2022.10007143.

N. Ammour, Y. Bazi, and N. Alajlan, "Multimodal Approach for Enhancing Biometric Authentication," J Imaging, vol. 9, no. 9, Sep. 2023, doi: 10.3390/jimaging9090168.

J. Mabe Parenreng et al., "INTERNATIONAL JOURNAL ON INFORMATICS VISUALIZATION journal homepage : www.joiv.org/index.php/joiv INTERNATIONAL JOURNAL ON INFORMATICS VISUALIZATION Solution for Public Smart Dispenser Using Digital Payment Based on the Fingerprint Minutiae Algorithm." [Online]. Available: www.joiv.org/index.php/joiv

B. H. Situmorang, “Identification of Biometrics Using Fingerprint Minutiae Extraction Based on Crossing Number Method,” Komputasi: Jurnal Ilmiah Ilmu Komputer dan Matematika, vol. 20, no. 1, pp. 71–80, Dec. 2022, doi: 10.33751/komputasi.v20i1.6814.

U. Sumalatha, K. Prakasha, S. Prabhu, and V. C. Nayak, "Multimodal biometric authentication: a novel deep learning framework integrating ECG, fingerprint, and finger knuckle print for high-security applications," Engineering Research Express, vol. 7, no. 1, Mar. 2025, doi: 10.1088/2631-8695/ad9aa0.

P. Nahar, N. Chaudhari, and S. Tanwani, "Fingerprint classification system using CNN," Multimed Tools Appl, vol. 81, pp. 24515–24527, 2022, doi: 10.1007/s11042-022-12294-4.

H. Zacarias, J. A. L. Marques, V. Felizardo, M. Pourvahab, and N. M. Garcia, "ECG Forecasting System Based on Long Short-Term Memory," Bioengineering, vol. 11, no. 1, Jan. 2024, doi: 10.3390/bioengineering11010089.

S. Mageshbabu and J. Mohana, "Enhanced ECG-Based Biometric Authentication using a Hybrid CNN-LSTM Framework," 2024 First International Conference on Software, Systems and Information Technology (SSITCON), pp. 1–7, 2024, doi: 10.1109/SSITCON62437.2024.10796214.

S. Son and K. Oh, "Integrated framework for estimating remaining useful lifetime through a deep neural network," Appl. Soft Comput., vol. 122, p. 108879, 2022, doi: 10.1016/j.asoc.2022.108879.

S. A. El-Rahman and A. S. Alluhaidan, "Enhanced multimodal biometric recognition systems based on deep learning and traditional methods in smart environments," PLoS One, vol. 19, no. 2 February, Feb. 2024, doi: 10.1371/journal.pone.0291084.

N. V Keerthana and M. Parimala Devi, "A Comprehensive Analysis of Minutiae Point Extraction in Biometric FingerPrint," in 2024 13th International Conference on System Modeling & Advancement in Research Trends (SMART), 2024, pp. 319–322. doi: 10.1109/SMART63812.2024.10882188.

Q. Huang, "Multimodal Biometrics Fusion Algorithm Using Deep Reinforcement Learning," Math Probl Eng, vol. 2022, 2022, doi: 10.1155/2022/8544591.

H. Zehir, T. Hafs, and S. Daas, "Healthcare Decision-Making with an ECG-Based Biometric System," 2023 International Conference on Decision Aid Sciences and Applications (DASA), pp. 88–92, 2023, doi: 10.1109/DASA59624.2023.10286620.