SENTIMENT ANALYSIS OF ICT SERVICE USER USING NAIVE BAYES CLASSIFIER AND SVM METHODS WITH TF-IDF TEXT WEIGHTING

Wulan Trisnawati; Arief Wibowo

doi:10.52436/1.jutif.2024.5.3.1784

Wulan Trisnawati Master of Computer Science, Faculty of Information Technology, Universitas Budi Luhur, Indonesia
Arief Wibowo Master of Computer Science, Faculty of Information Technology, Universitas Budi Luhur, Indonesia

DOI: https://doi.org/10.52436/1.jutif.2024.5.3.1784

Keywords: Data Mining, Naïve Bayes Classifier, Sentiment Analysis, Services, Support Vector Machine

Abstract

Pusintek is one of the government units in Indonesia responsible for managing Information and Communication Technology (ICT), providing various ICT services to users in central and regional offices through the ICT Service Catalog. The level of service fulfillment in Pusintek's IT Service Catalog significantly influences the effectiveness and efficiency in meeting service agreements, providing accurate information, and handling disruptions promptly. User satisfaction is measured through surveys to plan improvements to ICT services, but there is currently no method to classify sentiment from survey comment data. This research aims to classify sentiment and understand customer opinions and satisfaction trends regarding ICT services. The study applies the Naïve Bayes Classifier and Support Vector Machine (SVM) methods to classify positive and negative comments in user satisfaction surveys of ICT services. The data used consists of comments from the 2022 ICT user satisfaction survey results. Based on the test results, it is observed that the SVM algorithm provides higher accuracy compared to the Naïve Bayes algorithm. Utilizing the existing dataset with established opinion values, classification modeling using Naïve Bayes Classifier and Support Vector Machine (SVM) proves capable of classifying ICT user sentiment into 3 sentiment classes: Positive, Neutral, and Negative. From the data above, it is concluded that the SVM algorithm achieves the highest accuracy of 88.76%, highest precision of 89.68%, recall of 88.76%, and an f1-score of 89.12%.

Downloads

Download data is not yet available.

References

F. Valentinus, F. Sujono, I. Ariansyah, and D. A. H. Capah, “Implementation Of Data Mining With Classification And Forecasting Method Use Model Gaussian Naïve Bayes For Building Store (Studi Case: TB Sinar Jaya),” J. Tek. Inform., vol. 4, no. 2, pp. 413–420, 2023, doi: 10.52436/1.jutif.2023.4.2.701.

M. I. P. Hant and H. Hendry, “Data Mining Technique Using Naïve Bayes Algorithm To Predict Shopee Consumer Satisfaction Among Millennial Generation,” J. Tek. Inform., vol. 3, no. 4, pp. 829–838, 2022, doi: 10.20884/1.jutif.2022.3.4.295.

Y. Jumaryadi, M. Fajri, and B. Priambodo, “Evaluasi Kualitas Sistem Informasi Akademik Dengan Webqual dan IPA,” J. Inf. Syst., vol. 7, no. 2, pp. 99–106, 2022, doi: 10.33633/joins.v7i2.6187.

M. P. Cendana, H. Syafwan, and Rohminatin, “Application of Customer Relationship Management (CRM) To Increase Sales at UD Ulong Pian,” J. Tek. Inform., vol. 3, no. 3, pp. 543–552, 2022, doi: 10.20884/1.jutif.2022.3.3.238.

D. Fatmawati, W. Trisnawati, Y. Jumaryadi, and G. Triyono, “Klasifikasi Tingkat Kepuasan Penggunaan Layanan Teknologi Informasi Menggunakan Decision Tree,” KLIK Kaji. Ilm. Inform. dan Komput., vol. 3, no. 6, pp. 1056–1062, 2023, doi: 10.30865/klik.v3i6.803.

V. K. S. Que, A. Iriani, and H. D. Purnomo, “Analisis Sentimen Transportasi Online Menggunakan Support Vector Machine Berbasis Particle Swarm Optimization,” J. Nas. Tek. Elektro dan Teknol. Inf., vol. 9, no. 2, pp. 162–170, 2020, doi: 10.22146/jnteti.v9i2.102.

A. Nursalim and R. Novita, “Sentiment Analysis of Comments on Google Play Store, Twitter and Youtube To the Mypertamina Application With Support Vector Machine,” J. Tek. Inform., vol. 4, no. 6, pp. 1305–1312, 2023, doi: 10.52436/1.jutif.2023.4.4.801.

D. Taluke, R. S. M. Lakat, A. Sembel, E. Mangrove, and M. Bahwa, “Analisis Preferensi Masyarakat Dalam Pengelolaan Ekosistem Mangrove Di Pesisir Pantai Kecamatan Loloda Kabupaten Halmahera Barat,” Spasial, vol. 6, no. 2, pp. 531–540, 2019.

M. Ramzy and B. Ibrahim, “User satisfaction with Arabic COVID-19 apps: Sentiment analysis of users’ reviews using machine learning techniques,” Inf. Process. Manag., vol. 61, no. 3, p. 103644, 2024, doi: 10.1016/j.ipm.2024.103644.

M. P. Yuliani and I. N. Suarmanayasa, “Pengaruh Harga Dan Online Consumer Review terhadap Keputusan Pembelian Poduk pada Marketplace Tokopedia,” Prospek J. Manaj. dan Bisnis, vol. 3, no. 2, pp. 146–154, 2021.

R. Puspitasari, Y. Findawati, and M. A. Rosid, “Sentiment Analysis Of Post-Covid-19 Inflation Based On Twitter Using The K-Nearest Neighbor And Support Vector Machine Classification Methods,” J. Tek. Inform., vol. 4, no. 4, pp. 1–11, 2023, doi: 10.52436/1.jutif.2023.4.4.801.

L. N. K. Pasi and B. Sudaryanto, “Analisis Pengaruh Online Customer Reivews Dan Kualitas Pelayanan Terhadap Keputusan Pembelian Dengan Kepercayaan Sebagai Variabel Intervening (Studi Pada Konsumen Shopee Di Kota Semarang),” Diponegoro J. Manag., vol. 10, no. 3, pp. 1–12, 2021.

N. Colmekcioglu, R. Marvi, P. Foroudi, and F. Okumus, “Generation, susceptibility, and response regarding negativity: An in-depth analysis on negative online reviews,” J. Bus. Res., vol. 153, no. August, pp. 235–250, 2022, doi: 10.1016/j.jbusres.2022.08.033.

A. Muzaki and A. Witanti, “Sentiment Analysis of the Community in the Twitter To the 2020 Election in Pandemic Covid-19 By Method Naive Bayes Classifier,” J. Tek. Inform., vol. 2, no. 2, pp. 101–107, 2021, doi: 10.20884/1.jutif.2021.2.2.51.

Y. Yunitasari, A. Musdholifah, and A. K. Sari, “Sarcasm Detection For Sentiment Analysis in Indonesian Tweets,” IJCCS (Indonesian J. Comput. Cybern. Syst., vol. 13, no. 1, p. 53, 2019, doi: 10.22146/ijccs.41136.

G. Manik, I. Ernawati, and I. Nurlaili, “Analisis Sentimen Pada Review Pengguna E-Commerce Bidang Pangan Menggunakan Metode Support Vector Machine (Studi Kasus: Review Sayurbox dan Tanihub pada Google Play),” in Prosiding Seminar Nasional Mahasiswa Bidang Ilmu Komputer dan Aplikasinya, 2021, vol. 2, no. 2, pp. 64–74.

A. Isnanda, Y. Umaidah, and J. H. Jaman, “Implementasi Naïve Bayes Classifier Dan Information Gain Pada Analisis Sentimen Penggunaan E-Wallet Saat Pandemi,” J. Teknol. Inform. dan Komput., vol. 7, no. 2, pp. 144–153, 2021, doi: 10.37012/jtik.v7i2.648.

R. Kusnadi, Y. Yusuf, A. Andriantony, R. Ardian Yaputra, and M. Caintan, “Analisis Sentimen Terhadap Game Genshin Impact Menggunakan Bert,” Rabit J. Teknol. dan Sist. Inf. Univrab, vol. 6, no. 2, pp. 122–129, 2021, doi: 10.36341/rabit.v6i2.1765.

D. Septiani and I. Isabela, “Analisis Term Frequency Inverse Document Frequency (Tf-Idf) Dalam Temu Kembali Informasi Pada Dokumen Teks,” SINTESIA J. Sist. dan Teknol. Inf. Indones., vol. 1, no. 2, pp. 81–88, 2022.

S. M. Fani, R. Santoso, and S. Suparti, “Penerapan Text Mining Untuk Melakukan Clustering Data Tweet Akun Blibli Pada Media Sosial Twitter Menggunakan K-Means Clustering,” J. Gaussian, vol. 10, no. 4, pp. 583–593, 2021, doi: 10.14710/j.gauss.v10i4.30409.

Y. Cahyana and A. Mutoi Siregar, “Analisis Sentimen Pindah Ibu Kota Negara (IKN) Baru pada Twitter Menggunakan Algoritma Naive Bayes dan Support Vector Machine (SVM),” Fakt. Exacta, vol. 16, no. 3, pp. 1979–276, 2023, doi: 10.30998/faktorexacta.v16i3.16703.

H. Hermanto, A. Mustopa, and A. Y. Kuntoro, “Algoritma Klasifikasi Naive Bayes Dan Support Vector Machine Dalam Layanan Komplain Mahasiswa,” JITK (Jurnal Ilmu Pengetah. dan Teknol. Komputer), vol. 5, no. 2, pp. 211–220, 2020, doi: 10.33480/jitk.v5i2.1181.