PERFORMANCE COMPARISON OF SVM, NAIVE BAYES, AND LOGISTIC REGRESSION CLASSIFICATION ALGORITHMS IN ANALYZING NOICE APP USER REVIEWS

Ahmad Bahar; Tri Astuti; Primandani Arsi

doi:10.52436/1.jutif.2024.5.4.2061

Ahmad Bahar Informatics, Faculty of Computer Science, Universitas Amikom Purwokerto, Indonesia
Tri Astuti Informatics, Faculty of Computer Science, Universitas Amikom Purwokerto, Indonesia
Primandani Arsi Informatics, Faculty of Computer Science, Universitas Amikom Purwokerto, Indonesia

DOI: https://doi.org/10.52436/1.jutif.2024.5.4.2061

Keywords: Logistic Regression, Naive Bayes, Noice, SVM

Abstract

In the rapidly growing digital era, user reviews on distribution platforms such as the Google Play Store are a key indicator in assessing the popularity, quality, and user satisfaction of applications. This study aims to compare the performance of SVM, Naive Bayes, and Logistic Regression classification algorithms in analyzing user reviews of the Noice app, an audio content platform. The research involves steps such as data collection, data pre-processing, word embedding, modeling, model evaluation, and sentiment analysis. Testing was conducted using 1877 data. The data from the reviews were divided into scenarios, with training and testing data divided in ratios of 90:10, 80:20, and 70:30. The results showed that the SVM algorithm achieved the highest accuracy rate (80%) in the 90:10 data split scenario. However, Naive Bayes also showed competitive results with 78% accuracy in the same scenario. Meanwhile, Logistic Regression achieved 78% accuracy when the data was split in an 80:20 ratio. Evaluation was done using metrics such as accuracy, precision, recall, and F1-score. Sentiment analysis showed a positive trend with 1194 positive data compared to 683 negative data. From the comparison of data sharing scenarios and algorithms, SVM at 90:10 data sharing gave the best results.

Downloads

Download data is not yet available.

References

Noice, “Tentang Noice.” https://www.noice.id/tentang-noice/ (accessed Mar. 25, 2024).

C. Fikri, “Noice Dinobatkan Google Play Jadi Aplikasi Lokal Terkemuka,” Berita Satu, 2023. https://www.beritasatu.com/ototekno/1062758/noice-dinobatkan-google-play-jadi-aplikasi-lokal-terkemuka (accessed Mar. 25, 2024).

S. Vashishtha and S. Susan, “Fuzzy rule based unsupervised sentiment analysis from social media posts,” Expert Syst. Appl., vol. 138, p. 112834, 2019, doi: 10.1016/j.eswa.2019.112834.

E. Fitri, “Sentiment Analysis of the Ruangguru Application Using Naive Bayes, Random Forest and Support Vector Machine Algorithms,” J. Transform., vol. 18, no. 1, p. 71, 2020.

A. Saepudin, A. Faqih, and G. Dwilestari, “Perbandingan Algoritma Klasifikasi Support Vector Machine, Random Forest dan Logistic Regression Pada Ulasan Shopee,” J. TEKNO KOMPAK, vol. 18, no. 1, pp. 178–192, 2024.

A. Maulana, Inayah Khasnaputri Afifah, Asghafi Mubarrak, Kiagus Rachmat Fauzan, Ardhan Dwintara, and B. P. Zen, “Comparison of Logistic Regression, Multinomialnb, Svm, and K-Nn Methods on Sentiment Analysis of Gojek App Reviews on the Google Play Store,” J. Tek. Inform., vol. 4, no. 6, pp. 1487–1494, 2023, doi: 10.52436/1.jutif.2023.4.6.863.

I. Yunanto and S. Yulianto, “Twitter Sentiment Analysis Pedulilindungi Application Using Naïve Bayes and Support Vector Machine,” J. Tek. Inform., vol. 3, no. 4, pp. 807–814, 2022, doi: 10.20884/1.jutif.2022.3.4.292.

E. Suryati, Styawati, and A. Ari Aldino, “Analisis Sentimen Transportasi Online Menggunakan Ekstraksi Fitur Model Word2vec Text Embedding Dan Algoritma Support Vector Machine (SVM),” J. Teknol. dan Sist. Inf., vol. 4, no. 1, pp. 96–106, 2023.

M. Prasetya, M. Wulandari, and S. A. Nikmah, “Implementasi NLP (Natural Language Processing) Dasar pada Analisis Sentiment Review Spotify,” Stain. (Seminar Nas. Teknol. Sains), vol. 3, no. 1, pp. 145–153, 2024.

A. Lattu and D. Novira, “Analisis Sentimen Masyarakat Di Instagram Mengenai Kereta Cepat Jakarta-Bandung Menggunakan Algoritma Naïve Bayes Classifier,” Sismatik, vol. 1, pp. 1–8, 2023.

S. Wahyu Handani, D. Intan Surya Saputra, Hasirun, R. Mega Arino, and G. Fiza Asyrofi Ramadhan, “Sentiment analysis for go-jek on google play store,” J. Phys. Conf. Ser., vol. 1196, no. 1, 2019, doi: 10.1088/1742-6596/1196/1/012032.

Primandani Arsi, Pungkas Subarkah, and Bagus Adhi Kusuma, “Analisis Sentimen Game Genshin Impact pada Play Store Menggunakan Naïve Bayes Clasifier,” J. Ilm. Tek. Mesin, Elektro dan Komput., vol. 3, no. 1, pp. 161–170, 2023, doi: 10.51903/juritek.v3i1.1962.

H. Liu, X. Chen, and X. Liu, “A Study of the Application of Weight Distributing Method Combining Sentiment Dictionary and TF-IDF for Text Sentiment Analysis,” IEEE Access, vol. 10, pp. 32280–32289, 2022, doi: 10.1109/ACCESS.2022.3160172.

Imamah and F. H. Rachman, “Twitter sentiment analysis of Covid-19 using term weighting TF-IDF and logistic regresion,” Proceeding - 6th Inf. Technol. Int. Semin. ITIS 2020, pp. 238–242, 2020, doi: 10.1109/ITIS50118.2020.9320958.

R. Irwanda et al., “Sentiment analysis of Aceh tourism and its culture from Twitter data using support vector machine (SVM), naive Bayesian and k-nearest neighbour (KNN),” AIP Conf. Proc., vol. 3082, no. 1, 2024, doi: 10.1063/5.0202198.

I. Maulana, W. Apriandari, and A. Pambudi, “Analisis Sentimen Berbasis Aspek Terhadap Ulasan Aplikasi Mypertamina Menggunakan Support Vector Machine,” IDEALIS Indones. J. Inf. Syst., vol. 6, no. 2, pp. 172–181, 2023, doi: 10.36080/idealis.v6i2.3022.

S. H. Dinar Ajeng Kristiyanti, “Sentiment Analysis of Public Acceptance of Covid-19 Vaccines Types in Indonesia using Naïve Bayes, Support Vector Machine, and Long Short- Term Memory (LSTM),” vol. 5, no. 158, pp. 2–6, 2023.

A. Harun and D. Putri Ananda, “Analisa Sentimen Opini Publik Tentang Vaksinasi Covid-19 di Indonesia Menggunakan Naïve bayes dan Decission Tree,” MALCOM Indones. J. Mach. Learn. Comput. Sci., vol. 1, no. 1, pp. 58–64, 2021, doi: 10.57152/malcom.v1i1.63.

K. Hasanah, “Comparison of Sentiment Analysis Model for Shopee Comments on Google Play Store,” J. Sisfokom (Sistem Inf. dan Komputer), vol. 13, no. 1, pp. 21–30, 2024, doi: 10.32736/sisfokom.v13i1.1916.

R. I. Borman, Y. Fernando, and Y. Egi Pratama Yudoutomo, “Identification of Vehicle Types Using Learning Vector Quantization Algorithm with Morphological Features,” J. RESTI (Rekayasa Sist. dan Teknol. Informasi), vol. 6, no. 2, pp. 339–345, 2022, doi: 10.29207/resti.v6i2.3954.

A. I. KABIR, K. AHMED, and R. KARIM, “Word Cloud and Sentiment Analysis of Amazon Earphones Reviews with R Programming Language,” Inform. Econ., vol. 24, no. 4/2020, pp. 55–71, 2020, doi: 10.24818/issn14531305/24.4.2020.05.

R. A. Husen, R. Astuti, L. Marlia, R. Rahmaddeni, and L. Efrizoni, “Analisis Sentimen Opini Publik pada Twitter Terhadap Bank BSI Menggunakan Algoritma Machine Learning,” MALCOM Indones. J. Mach. Learn. Comput. Sci., vol. 3, no. 2, pp. 211–218, 2023, doi: 10.57152/malcom.v3i2.901.

R. A. Fauzianto et al., “Analisis Sentimen Opini Masyarakat Terhadap Tech Winter Pada Twitter,” vol. 4, no. 9, pp. 1577–1585, 2023, doi: 10.46799/jsa.v3i9.909