Optimizing Naive Bayes for Sentiment Analysis of M-Passport Reviews Using N-Gram and Synthetic Minority Over-sampling Technique

Devia Kartika; Sarjon Defit

doi:10.52436/1.jutif.2026.7.3.5853

Authors

Devia Kartika Universitas Putra Indonesia YPTK Padang, Indonesia
Sarjon Defit Universitas Putra Indonesia YPTK Padang, Indonesia

DOI:

https://doi.org/10.52436/1.jutif.2026.7.3.5853

Keywords:

Digital Public Services, M-Passport, N-Gram, Naïve Bayes, Sentiment Analysis, SMOTE, TF-IDF

Abstract

The diverse user perceptions and increasing number of negative reviews of the M-Passport application indicate the need for sentiment analysis-based evaluation to more accurately measure the quality of digital immigration services. This study aims to analyze user sentiment towards the M-Passport application using an optimized Naïve Bayes classification model. Review data was obtained through web scraping from various digital platforms and processed using text preprocessing, TF-IDF feature extraction, N-Gram representation, and the Synthetic Minority Over-sampling Technique (SMOTE) technique to address data representativeness. The proposed model classifies user reviews into positive, neutral, and negative sentiment categories. Test results show that optimization using N-Gram and SMOTE successfully improved model performance, with accuracy increasing from 61% to 77.51%, precision from 0.75 to 0.78, recall from 0.53 to 0.78, and F1-score from 0.50 to 0.77. These results demonstrate that the combination of feature engineering and data balancing can improve text context representation and sentiment classification stability across multiple classes. Furthermore, sentiment analysis successfully identified key factors contributing to user dissatisfaction, such as technical constraints, feature limitations, and application difficulty. These results demonstrate that the proposed approach is effective in supporting data-driven evaluation to improve the quality of digital immigration services.

Downloads

Download data is not yet available.

References

A. Pujianto, A. Mulyati, And R. Novaria, “Pemanfaatan Big Data Dan Perlindungan Privasi Konsumen Di Era Ekonomi Digital Agung Pujianto 1) , Awin Mulyati 2) , Rachmawati Novaria 3),” Majalah Ilmiah Bijak, Vol. 127, No. 2, Pp. 127–137, 2018, [Online]. Available: Http://Ojs.Stiami.Ac.Id | https://doi.org/10.31334/bijak.v15i2.201

S. Mufti Prasetiyo, R. Gustiawan, And F. Rizzel Albani, “Biikma : Buletin Ilmiah Ilmu Komputer Dan Multimedia Analisis Pertumbuhan Pengguna Internet Di Indonesia,” Vol. 2, No. 1, 2024, [Online]. Available: Https://Jurnalmahasiswa.Com/Index.Php/Biikma| | https://doi.org/10.31454/biikma.v15i2.321

Deandlles Christover, Aji Syarif Hidayattulah, And Indah Mawarni, “Penerapan Konsep-Konsep Digitalisasi Dalam Pelayanan Publik Di Kantor Desa Manunggal Jaya Kecamatan Tenggarong Seberang Kabupaten Kutai Kartanegara,” Journal Of Research And Development On Public Policy, Vol. 2, No. 2, Pp. 199–214, 2023, https://doi.org/10.58684/jarvic.v2i2.73

A. Zahra, R. Mayasari, And I. Pernamasari, “Analisis Sentimen Pada Aplikasi M-Paspor Menggunakan Algoritma Naïve Bayes Classifier,” Action Research Literate, Vol. 8, No. 8, Pp. 2365–2371, 2024, https://doi.org/10.46799/arl.v8i8.466

D. A. Putri, D. A. Kristiyanti, E. Indrayuni, A. Nurhadi, And D. R. Hadinata, “Comparison Of Naive Bayes Algorithm And Support Vector Machine Using Pso Feature Selection For Sentiment Analysis On E-Wallet Review,” J. Phys. Conf. Ser., Vol. 1641, No. 1, 2020, https://doi.org/10.1088/1742-6596/1641/1/012085

M. H. S. Quadri* And Dr. R. K. Selvakumar, “Performance Of Naïve Bayes In Sentiment Analysis Of User Reviews Online,” International Journal Of Innovative Technology And Exploring Engineering, Vol. 10, No. 2, Pp. 64–68, 2020, https://doi.org/10.35940/ijitee.a8198.1210220

P. D. P. Silitonga, M. Hasibuan, Z. Situmorang, And D. Purba, “Comparison Of Tiktok User Sentiment Analysis Accuracy With Naïve Bayes And Support Vector Machine,” International Journal Of Advanced Trends In Computer Science And Engineering, Vol. 12, No. 1, Pp. 11–15, 2023, https://doi.org/10.30534/ijatcse/2023/031212023

M. Horvat, G. Gledec, And F. Leontić, “Hybrid Natural Language Processing Model For Sentiment Analysis During Natural Crisis,” Electronics (Switzerland), Vol. 13, No. 10, May 2024, https://doi.org/10.3390/electronics13101991

R. A. Siregar, Y. A. Sari, And I. Indriati, “Analisis Sentimen Kebijakan New Normal Dengan Menggunakan Automated Lexicon Senti N-Gram,” Jurnal Teknologi Informasi Dan Ilmu Komputer, Vol. 10, No. 1, Pp. 29–34, Feb. 2023, https://doi.org/.25126/jtiik.2023105006

M. E. Purbaya, D. P. Rakhmadani, M. P. Arum, And L. Z. Nasifah, “Implementation Of N-Gram Methodology To Analyze Sentiment Reviews For Indonesian Chips Purchases In Shopee E-Marketplace,” Jurnal Resti, Vol. 7, No. 3, Pp. 609–617, Jun. 2023, https://doi.org/10.29207/resti.v7i3.4726

A. M. Priyatno And F. I. Firmananda, “N-Gram Feature For Comparison Of Machine Learning Methods On Sentiment In Financial News Headlines,” Riggs: Journal Of Artificial Intelligence And Digital Business, Vol. 1, No. 1, Pp. 01–06, Jul. 2022, https://doi.org/10.31004/riggs.v1i1.4

D. Normawati And S. A. Prayogi, “Implementasi Naïve Bayes Classifier Dan Confusion Matrix Pada Analisis Sentimen Berbasis Teks Pada Twitter,” Jurnal Sains Komputer & Informatika (J-Sakti, Vol. 5, No. 2, Pp. 697–711, 2021. http://dx.doi.org/10.30645/j-sakti.v5i2.369

L. Hermawan, M. B. Ismiati, J. Bangau, N. 60, And M. Charitas, “Pembelajaran Text Preprocessing Berbasis Simulator Untuk Mata Kuliah Information Retrieval,” Transformatika, Vol. 17, No. 2, Pp. 188–199, 2020. https://doi.org/10.26623/transformatika.v17i2.1705

H. Ma’rifah, A. Prasetya Wibawa, And M. I. Akbar, “Sains, Aplikasi, Komputasi Dan Teknologi Informasi Klasifikasi Artikel Ilmiah Dengan Berbagai Skenario Preprocessing,” Vol. 2, No. 2, P. 70, 2020. http://dx.doi.org/10.30872/jsakti.v2i2

S. A. H. Bahtiar, C. K. Dewa, And A. Luthfi, “Comparison Of Naïve Bayes And Logistic Regression In Sentiment Analysis On Marketplace Reviews Using Rating-Based Labeling,” Journal Of Information Systems And Informatics, Vol. 5, No. 3, Pp. 915–927, 2023, https://doi.org/10.51519/journalisi.v5i3.539

Pristiyono, M. Ritonga, M. A. Al Ihsan, A. Anjar, And F. H. Rambe, “Sentiment Analysis Of Covid-19 Vaccine In Indonesia Using Naïve Bayes Algorithm,” Iop Conf. Ser. Mater. Sci. Eng., Vol. 1088, No. 1, P. 012045, 2021, , https://doi.org/10.1088/1757-899x/1088/1/012045

T. Chamidy, “Application Of Smote In Sentiment Analysis Of Myxl User Reviews On Google Play Store,” 2025. https://doi.org/10.14421/jiska.2025.10.1.74-86

L. D. Cahya, A. Luthfiarta, J. I. T. Krisna, S. Winarno, And A. Nugraha, “Improving Multi-Label Classification Performance On Imbalanced Datasets Through Smote Technique And Data Augmentation Using Indobert Model,” Jurnal Nasional Teknologi Dan Sistem Informasi, Vol. 9, No. 3, Pp. 290–298, Jan. 2024, http://dx.doi.org/10.25077/Teknosi.V9i3.2023.290-298

O. Rainio, J. Teuho, And R. Klén, “Evaluation Metrics And Statistical Tests For Machine Learning,” Sci. Rep., Vol. 14, No. 1, Dec. 2024, Doi: 10.1038/S41598-024-56706-X.

A. R. Lubis, Y. Y. Lase, D. A. Rahman, And D. Witarsyah, “Improving Spell Checker Performance For Bahasa Indonesia Using Text Preprocessing Techniques With Deep Learning Models,” Ingenierie Des Systemes D’information, Vol. 28, No. 5, Pp. 1335–1342, 2023, https://doi.org/10.18280/isi.280522

M. U. Albab, Y. K. P., And M. N. Fawaiq, “Optimization Of The Stemming Technique On Text Preprocessing President 3 Periods Topic,” Jurnal Transformatika, Vol. 20, No. 2, Pp. 1–12, Jan. 2023, https://doi.org/10.26623/Transformatika.V20i2.5374

Rianto, A. B. Mutiara, E. P. Wibowo, And P. I. Santosa, “Improving The Accuracy Of Text Classification Using Stemming Method, A Case Of Non-Formal Indonesian Conversation,” J. Big Data, Vol. 8, No. 1, Dec. 2021, https://doi.org/10.1186/S40537-021-00413-1

S. Khairunnisa, A. Adiwijaya, And S. Al Faraby, “Pengaruh Text Preprocessing Terhadap Analisis Sentimen Komentar Masyarakat Pada Media Sosial Twitter (Studi Kasus Pandemi Covid-19),” Jurnal Media Informatika Budidarma, Vol. 5, No. 2, P. 406, Apr. 2021, https://doi.org/10.30865/Mib.V5i2.2835

R. B. Hadiprakoso, H. Setiawan, R. N. Yasa, And Girinoto, “Text Preprocessing For Optimal Accuracy In Indonesian Sentiment Analysis Using A Deep Learning Model With Word Embedding,” Aip Conf. Proc., Vol. 2680, No. 1, P. 020050, Dec. 2023, https://doi.org/10.1063/5.0126116

A. F. Aufar, Mochamad Alfan Rosid, A. Eviyanti, And I. R. I. Astutik, “Optimizing Text Preprocessing For Accurate Sentiment Analysis On E-Wallet Reviews,” Jicte (Journal Of Information And Computer Technology Education), Vol. 7, No. 2, Pp. 42–50, Oct. 2023, https://doi.org/10.21070/Jicte.V7i2.1650

D. Wulan Yekti Rahayu Et Al., “Performance Of Machine Learning Algorithms On Imbalanced Sentiment Datasets Without Balancing Techniques,” 2025. [Online]. Http://Jurnal.Polibatam.Ac.Id/Index.Php/Jaic | https://doi.org/10.30871/jaic.v9i3.9584

R. M. Prabha And S. Sasikala, “Data Analytics For Imbalanced Dataset,” Journal Of Computer Science, Vol. 20, No. 2, Pp. 207–217, 2024, Doi: 10.3844/Jcssp.2024.207.217.

M. R. Ramadhan And K. Budiman, “Sentiment Analysis Of Presidential Candidates In 2024: A Comparison Of The Performance Of Support Vector Machine And Random Forest With N-Gram Method,” Recursive Journal Of Informatics, Vol. 3, No. 1, Pp. 34–42, Mar. 2025, https://doi.org/10.15294/Rji.V3i1.8385

M. B. Alfarazi, M. ’Ariful Furqon, And H. Soepandi, “Sentiment Analysis Of Universitas Jember’s Sister For Student Application Using Gaussian Naive Bayes And N-Gram,” Journal Of Innovation Information Technology And Application (Jinita), Vol. 6, No. 2, Pp. 101–108, Dec. 2024, https://doi.org/10.35970/Jinita.V6i2.2400

H. Noer Rofiq And G. Mafela Mutiara Sujak, “Analisis Persepsi Masyarakat Terhadap Lelang Indonesia Melalui Analisis N-Gram Dan Sentimen,” Vol. 7, 2024, [Online]. Available: https://doi.org/10.31598.