Comparative Evaluation and Hyperparameter Optimization of Random Forest and Extreme Gradient Boosting for Phishing Web Address Detection Using Lexical and Host-Based Features
DOI:
https://doi.org/10.52436/1.jutif.2026.7.5.5760Keywords:
Ensemble Learning, Host-Base Feature, Hyperparameter Optimization, Lexical Features, Phising URL DetectionAbstract
This study addresses the growing threat of phishing attacks, particularly those exploiting malicious URLs that mimic legitimate websites and bypass traditional rule-based or blacklist detection systems. The purpose of this research is to conduct a comparative analysis and optimization of ensemble learning models for phishing URL detection using a combination of lexical and host-based features. The proposed approach involves feature extraction from URL structures and domain-related attributes, followed by classification using Random Forest and XGBoost. Hyperparameter optimization is performed using Grid Search to obtain the best model configuration. Model performance is evaluated using Stratified 5-Fold Cross-Validation, with metrics including accuracy and Area Under the ROC Curve (AUC), along with statistical significance testing. The experimental results show that the optimized Random Forest model achieves the best performance with an accuracy of approximately 0.97, slightly outperforming XGBoost with an accuracy of 0.96. Statistical analysis using a paired t-test indicates that this difference is significant (p = 0.000615). The results also demonstrate that combining lexical and host-based features significantly improves classification performance.This study provides empirical evidence that optimized ensemble learning models with hybrid feature representation can effectively detect phishing URLs with high accuracy and stability. The findings contribute to the development of more reliable and robust phishing detection systems in real-world cybersecurity environments.
Downloads
References
H. Jabbar and S. Al-Janabi, “AI-Driven Phishing Detection: Enhancing Cybersecurity with Reinforcement Learning,” Journal of Cybersecurity and Privacy, vol. 5, no. 2, Jun. 2025, doi: 10.3390/jcp5020026.
P. Maneriker, J. W. Stokes, E. G. Lazo, D. Carutasu, F. Tajaddodianfar, and A. Gururajan, “URLTran: Improving Phishing URL Detection Using Transformers,” Jun. 2021, doi: 10.1109/MILCOM52596.2021.9653028.
Y. S. M. G. Yeremia, D. K. -, L. K. D. -, and A. K. N. -, “SISTEM KLASIFIKASI URL UNTUK DETEKSI SITUS BERBAHAYA BERBASIS ANALISIS FITUR JARINGAN,” Jurnal Informatika dan Teknik Elektro Terapan, vol. 14, no. 1, Jan. 2026, doi: 10.23960/jitet.v14i1.8186.
A. F. Mahmud and S. Wirawan, “Sistemasi: Jurnal Sistem Informasi Deteksi Phishing Website menggunakan Machine Learning Metode Klasifikasi Phishing Website Detection using Machine Learning Classification Method.” [Online]. Available: http://sistemasi.ftik.unisi.ac.id
D. T. Mosa, M. Y. Shams, A. A. Abohany, E. S. M. El-Kenawy, and M. Thabet, “Machine Learning Techniques for Detecting Phishing URL Attacks,” Computers, Materials and Continua, vol. 75, no. 1, pp. 1271–1290, 2023, doi: 10.32604/cmc.2023.036422.
A. K. Jain and B. B. Gupta, “Phishing detection: Analysis of visual similarity based approaches,” 2017, Hindawi Limited. doi: 10.1155/2017/5421046.
S. Kapan and E. Sora Gunal, “Improved Phishing Attack Detection with Machine Learning: A Comprehensive Evaluation of Classifiers and Features,” Applied Sciences (Switzerland), vol. 13, no. 24, Dec. 2023, doi: 10.3390/app132413269.
P. Xu, “A Transformer-based Model to Detect Phishing URLs,” Sep. 2021, [Online]. Available: http://arxiv.org/abs/2109.02138
S. Aung, ) Chaw, T. Zan, and H. Yamana, “A Survey of URL-based Phishing Detection.” [Online]. Available: http://quadrodeofertas.com.br/www1.
Hanif Abdul Karim Afandi, M. Lazaro Fa. Al-Dzaki, Nurul Qomariasih, and Reza Aulia Wildana, “GuardSurfing : Ekstensi Browser sebagai Alat Bantu Deteksi Website Phishing dengan Metode Klasifikasi XGBoost untuk Deteksi URL Phishing Berbasis Flask Framework,” Info Kripto, vol. 19, no. 2, pp. 73–85, Sep. 2025, doi: 10.56706/ik.v19i2.124.
Nailah Azzahra, Merry Dwi Handayani, and Awwaliyah Aliyah, “Evaluasi Kinerja AI berbasis Recurrent Neural Network (RNN) dalam Mengidentifikasi Ancaman Phising pada URL Website,” Bridge : Jurnal Publikasi Sistem Informasi dan Telekomunikasi, vol. 3, no. 3, pp. 15–37, Jun. 2025, doi: 10.62951/bridge.v3i3.485.
Y. Dang, Z. Chen, H. Li, and H. Shu, “A Comparative Study of non-deep Learning, Deep Learning, and Ensemble Learning Methods for Sunspot Number Prediction,” Applied Artificial Intelligence, vol. 36, no. 1, 2022, doi: 10.1080/08839514.2022.2074129.
A. Sayed Abdou and N. Ramadan Darwish, “Early Prediction of Software Defect using Ensemble Learning: A Comparative Study,” Egypt, 2021.
O. K. Sahingoz, E. Buber, O. Demir, and B. Diri, “Machine learning based phishing detection from URLs,” Expert Syst. Appl., vol. 117, pp. 345–357, Mar. 2019, doi: 10.1016/j.eswa.2018.09.029.
K. Haynes, H. Shirazi, and I. Ray, “Lightweight URL-based phishing detection using natural language processing transformers for mobile devices,” in Procedia Computer Science, Elsevier B.V., 2021, pp. 127–134. doi: 10.1016/j.procs.2021.07.040.
E. S. Shombot, G. Dusserre, R. Bestak, and N. B. Ahmed, “An application for predicting phishing attacks: A case of implementing a support vector machine learning model,” Cyber Security and Applications, vol. 2, Jan. 2024, doi: 10.1016/j.csa.2024.100036.
E. Ismanto, A. Fadlil, A. Yudhana, and K. Kitagawa, “A Comparative Study of Improved Ensemble Learning Algorithms for Patient Severity Condition Classification,” Journal of Electronics, Electromedical Engineering, and Medical Informatics, vol. 6, no. 3, pp. 312–321, Jul. 2024, doi: 10.35882/jeeemi.v6i3.452.
T. Kurniawan, L. Hermawanti, and A. N. Safriandono, “Interpretable Machine Learning with SHAP and XGBoost for Lung Cancer Prediction Insights,” 2024. [Online]. Available: http://jurnal.polibatam.ac.id/index.php/JAIC
B. K. Gontla, P. Gundu, P. J. Uppalapati, K. Narasimharao, and S. M. Hussain, “A Machine Learning Approach to Identify Phishing Websites: A Comparative Study of Classification Models and Ensemble Learning Techniques,” EAI Endorsed Transactions on Scalable Information Systems, vol. 10, no. 5, pp. 1–9, 2023, doi: 10.4108/eetsis.vi.3300.
B. K. Gontla, P. Gundu, P. J. Uppalapati, K. Narasimharao, and S. M. Hussain, “A Machine Learning Approach to Identify Phishing Websites: A Comparative Study of Classification Models and Ensemble Learning Techniques,” EAI Endorsed Transactions on Scalable Information Systems, vol. 10, no. 5, pp. 1–9, 2023, doi: 10.4108/eetsis.vi.3300.
A. U. Z. Asif, H. Shirazi, and I. Ray, “Machine Learning-Based Phishing Detection Using URL Features: A Comprehensive Review,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer Science and Business Media Deutschland GmbH, 2023, pp. 481–497. doi: 10.1007/978-3-031-44274-2_36.
S. Günal, “Hybrid feature selection for text classification,” Turkish Journal of Electrical Engineering and Computer Sciences, vol. 20, no. SUPPL.2, pp. 1296–1311, 2012, doi: 10.3906/elk-1101-1064.
M. Zaffar et al., “A hybrid feature selection framework for predicting students performance,” Computers, Materials and Continua, vol. 70, no. 1, pp. 1893–1920, 2021, doi: 10.32604/cmc.2022.018295.
M. Sánchez-Paniagua, E. Fidalgo, E. Alegre, and R. Alaiz-Rodríguez, “Phishing websites detection using a novel multipurpose dataset and web technologies features,” Expert Syst. Appl., vol. 207, Nov. 2022, doi: 10.1016/j.eswa.2022.118010.
M. Adi, S. Aritonang, M. J. Simanulang, T. P. Batubara, I. Zega, and H. Afrizal, “Optimasi Natural Languange Processing (NLP) dan Support Vector Machine (SVM) Dalam Deteksi Phishing URL Website,” Jurnal Teknik Informatika (JUTIF), vol. 6, no. 6, 2025, doi: 10.52436/1.jutif.6.6.3540.
M. A. Uddin, M. Mahiuddin, and I. H. Sarker, “An Explainable Transformer-based Model for Phishing Email Detection: A Large Language Model Approach,” Aug. 2025, [Online]. Available: http://arxiv.org/abs/2402.13871
M. Abdolrazzagh-Nezhad and N. Langarib, “Phishing Detection Techniques: A review,” Data Science: Journal of Computing and Applied Informatics, vol. 9, no. 1, pp. 32–46, Jan. 2025, doi: 10.32734/jocai.v9.i1-19904.
E. M. Ferrouhi and I. Bouabdallaoui, “A comparative study of ensemble learning algorithms for high-frequency trading,” Sci. Afr., vol. 24, Jun. 2024, doi: 10.1016/j.sciaf.2024.e02161.
G. Zhang, C. Wu, and J. Wang, “Reinforcement Learning-Enhanced Anti-Phishing Method with Adversarial Edge Collaboration for Smart Grid IoT Networks,” International Journal of High Speed Electronics and Systems, Aug. 2025, doi: 10.1142/s0129156425409180.
A. Butnaru, A. Mylonas, and N. Pitropakis, “Towards lightweight url-based phishing detection,” Future Internet, vol. 13, no. 6, Jun. 2021, doi: 10.3390/fi13060154.
L. Thaqi, A. Halili, K. Vishi, and B. Rexha, “NoPhish: Efficient Chrome Extension for Phishing Detection Using Machine Learning Techniques,” Sep. 2024, [Online]. Available: http://arxiv.org/abs/2409.10547
E. Benavides-Astudillo, W. Fuertes, S. Sanchez-Gordon, D. Nuñez-Agurto, and G. Rodríguez-Galán, “A Phishing-Attack-Detection Model Using Natural Language Processing and Deep Learning,” Applied Sciences (Switzerland), vol. 13, no. 9, May 2023, doi: 10.3390/app13095275.
E. V. Anthony and S. P. Barus, “Perbandingan Algoritma Machine Learning dan Deep Learning Dalam Analisis Sentimen Komentar Video YouTube Berjudul Hidup Tanpa Sosial Media,” JIKO (Jurnal Informatika dan Komputer), vol. 9, no. 2, p. 370, Jun. 2025, doi: 10.26798/jiko.v9i2.1619.
L. N. Hapsari, I. Fannani, Y. Rahmawati, and A. Muhariya, “Explainable Machine Learning untuk Prediksi Risiko Penyakit Jantung Menggunakan Random Forest dan Analisis SHAP,” remik, vol. 10, no. 1, pp. 190–199, Jan. 2026, doi: 10.33395/remik.v10i1.15766.
W. Bambang Triadi Handaya, “Lukito, Deteksi Website Phishing Menggunakan Teknik Machine Learning 69 Deteksi Website Phishing Menggunakan Teknik Machine Learning,” 2023.
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Mhd Adi Setiawan Aritonang, Nugroho Syahputra, Noorhamizah Mohamed Nasir, Ardianto Pranata, Toras Pangidoan Batubara, Vadly Syafril, Rudi Purwana, Muhammad Nazwan

This work is licensed under a Creative Commons Attribution 4.0 International License.

</a



