Sentiment Analysis of Fizzo Novel Application Using Support Vector Machine and Naïve Bayes Algorithm with SEMMA Framework

Satrio  Pambudi; Pratomo Setiaji; Wiwit Agus  Triyanto

doi:10.52436/1.jutif.2025.6.4.4875

Authors

Satrio Pambudi Information System, Universitas Muria Kudus, Indonesia
Pratomo Setiaji Information System, Universitas Muria Kudus, Indonesia
Wiwit Agus Triyanto Information System, Universitas Muria Kudus, Indonesia

DOI:

https://doi.org/10.52436/1.jutif.2025.6.4.4875

Keywords:

Naïve Bayes, SEMMA, Sentiment Analysis, SMOTE, Support Vector Machine, TF-IDF

Abstract

The increasing popularity of digital reading platforms in Indonesia, such as Fizzo Novel, has generated many user reviews that can be analyzed to understand their satisfaction. This study analyzes user sentiment toward Fizzo Novel using the SEMMA (Sample, Explore, Modify, Model, Assess) framework, and compares the performance of the Support Vector Machine (SVM) and Naïve Bayes algorithms. A total of 139,759 reviews were collected from the Google Play Store through web scraping. The data was then processed through normalization, tokenization, lexicon-based sentiment labeling, and feature extraction using TF-IDF. To address class imbalance, the SMOTE technique was applied. The results showed that SVM achieved the highest accuracy, exceeding 96%, with a consistent F1-score across all sentiment classes. In contrast, Naïve Bayes recorded lower accuracy (75.82% before SMOTE and 73.63% after SMOTE), along with a decline in performance for the neutral class. SVM proved more reliable in handling large and imbalanced text data. Practically, the results of this study can help application developers such as Fizzo Novel in automatically understanding user opinions. With an accurate sentiment classification model, developers can monitor reviews in real-time, identify issues such as excessive advertising or an unpopular chapter division system, and design feature improvements based on real user needs. This research also provides a foundation for algorithm selection in future large-scale sentiment analysis projects and recommends SVM as the more appropriate choice in this context.

Downloads

Download data is not yet available.

References

K. G. Segara and M. I. P. Nasution, “Perkembangan Teknologi Informasi di Indonesia: Tantangan dan Peluang,” Media Mhs. Indones., vol. 3, no. 1, pp. 21–33, 2025, doi: https://doi.org/10.61722/jssr.v3i1.3128.

B. A. Diana and J. A. Sari, “Dampak Transformasi Digitalisasi terhadap Perubahan Perilaku Masyarakat Pedesaan,” J. Pemerintah. dan Polit., vol. 9, no. 2, pp. 88–96, 2024, doi: 10.36982/jpg.v9i2.3896.

I. P. Haris, Y. I. N. Setiawan, R. Rendi, and N. K. Fajarwati, “Tren Terkini Dalam Ilmu Komunikasi Di Indonesia: Antara Transformasi Digital Dan Dinamika Budaya,” Filos. Publ. Ilmu Komunikasi, Desain, Seni Budaya, vol. 1, no. 1, pp. 140–149, 2024, doi: https://doi.org/10.62383/filosofi.v1i1.73.

I. Suryawati and S. Alam, “Transformasi Media Cetak Ke Platform Digital (Analisis Mediamorfosis Harian Solopos),” J. Signal, vol. 10, no. 2, pp. 177–361, 2022, doi: 10.33603/signal.v10i2.7240.

Sumarni, A. Ambarwati, and M. Badrih, “Pemanfaatan Spotify Sebagai Media Dongeng dalam Upaya Digitalisasi Sastra Anak,” Didakt. J. Kependidikan, vol. 13, no. 1, pp. 251–260, 2024, doi: https://doi.org/10.58230/27454312.408.

T. Arlovin, Kusrini, and Kusnawi, “Analisis Sentimen Review Pengguna Aplikasi Fizzo Novel Di Google Play Menggunakan Algoritma Naive Bayes,” J. Inform. Teknol. dan Sains, vol. 6, no. 1, pp. 65 – 70, 2024, doi: 10.51401/jinteks.v6i1.3909.

S. N. Adhan, G. N. A. Wibawa, D. C. Arisona, I. Yahya, Agusrawati, and Ruslan, “Analisis Sentimen Ulasan Aplikasi Wattpad Di Google Play Store Dengan Metode Random Forest,” AnoaTIK J. Teknol. Inf. dan Komput., vol. 2, no. 1, pp. 6–15, 2024, doi: 10.33772/anoatik.v2i1.32.

A. R. A. Baso, Budiandriani, Ramlawati, and Mahfudnurnajamuddin, “Analisis Komprehensif tentang Strategi Pemasaran yang Beretika dan Keterlibatan Pelanggan dalam Bisnis TikTok,” SEIKO J. Manag. Bus., vol. 7, no. 1, pp. 654–662, 2024, doi: https://doi.org/10.37531/sejaman.v7i1.6922.

E. Suhadi, “Analisis Sentimen Aplikasi Bisa Ekspor Pada Ulasan Pengguna Di Google Play Dengan Naive Bayes,” JIKA (Jurnal Inform. Univ. Muhammadiyah Tangerang, vol. 9, no. 1, pp. 93–101, 2025, doi: http://dx.doi.org/10.31000/jika.v9i1.12876.

A. Syafa’aturrohman, O. Nurdiawan, F. M. Basysyar, and M. Sulaeman, “Naive Bayes Meningkatkan Model Analisis Sentimen Pada Ulasan Aplikasi DANA Di Playstore Indonesia,” NFORMATION Manag. Educ. Prof., vol. 9, no. 2, pp. 171–180, 2024, doi: https://doi.org/10.51211/imbi.v9i2.3330.

B. A. Maulana, M. J. Fahmi, A. M. Imran, and N. Hidayati, “Analisis Sentimen Terhadap Aplikasi Pluang Menggunakan Algoritma Naive Bayes dan Support Vector Machine (SVM),” MALCOM Indones. J. Mach. Learn. Comput. Sci., vol. 4, no. 2, pp. 375–384, 2024, doi: 10.57152/malcom.v4i2.1206.

Styawati, A. R. Isnain, N. Hendrastuty, and L. Andraini, “Comparison of Support Vector Machine and Naïve Bayes on Twitter Data Sentiment Analysis,” J. Inform. J. Pengemb. IT, vol. 6, no. 1, pp. 56–60, 2021, doi: 10.30591/jpit.v6i1.3245.

U. H. Laksono and R. R. Suryono, “Sentiment Analysis Of Online Dating Apps Using Support Vector Machine And Naïve Bayes Algorithms,” J. Tek. Inform., vol. 6, no. 1, pp. 229–238, 2025, doi: https://doi.org/10.52436/1.jutif.2025.6.1.2105.

A. A. Baskara, N. M. Piranti, and M. F. Romdendine, “Framework Data Mining : Sebuah Survei,” JATI (Jurnal Mhs. Tek. Inform., vol. 9, no. 3, pp. 4886–4895, 2025, doi: https://doi.org/10.36040/jati.v9i3.13803.

O. Firas, “A combination of SEMMA & CRISP-DM models for effectively handling big data using formal concept analysis based knowledge discovery: A data mining approach,” World J. Adv. Eng. Technol. Sci., vol. 8, no. 1, pp. 009–014, 2023, doi: 10.30574/wjaets.2023.8.1.0147.

R. H. Hafizh, “Pengembangan Chatbot Berbasis Jaringan Saraf Transformer Untuk Layanan Informasi Akademik Dan Keuangan Mahasiswa Di Universitas Muhammadiyah Sukabumi,” JITET (Jurnal Inform. dan Tek. Elektro Ter., vol. 12, no. 3, pp. 3412–3419, 2024, doi: http://dx.doi.org/10.23960/jitet.v12i3.5002 PENGEMBANGAN.

K. Irfansyah and Z. Fatah, “Implementasi Algoritma Clustering K-Means Pada Pengguna Wartel Di Pondok Pesantren Salafiyah Syafi ’ Iyah Sukorejo,” J. Ilm. MULTIDISIPLIN ILMU, vol. 1, no. 5, pp. 81–86, 2024, doi: https://doi.org/10.69714/55xet429 IMPLEMENTASI.

G. Tamami, W. A. Triyanto, and S. Muzid, “Sentiment Analysis Mobile JKN Reviews Using SMOTE Based LSTM,” IJCCS (Indonesian J. Comput. Cybern. Syst., vol. 19, no. 1, pp. 13–24, 2024, doi: https://doi.org/10.22146/ijccs.101910.

D. Rifaldi, A. Fadlil, and Herman, “Teknik Preprocessing Pada Text Mining Menggunakan Data Tweet ‘Mental Health,’” Decod. J. Pendidik. Teknol. Inf., vol. 3, no. 2, pp. 161–171, 2023, doi: 10.51454/decode.v3i2.131.

J. E. Br Sinulingga and H. C. K. Sitorus, “Analisis Sentimen Opini Masyarakat terhadap Film Horor Indonesia Menggunakan Metode SVM dan TF-IDF,” J. Manaj. Inform., vol. 14, no. 1, pp. 42–53, 2024, doi: 10.34010/jamika.v14i1.11946.

A. S. Firmansyah, A. Aziz, and M. Ahsan, “Optimasi K-Nearest Neighbor Menggunakan Algoritma Smote Untuk Mengatasi Imbalance Class Pada Klasifikasi Analisis Sentimen,” JATI (Jurnal Mhs. Tek. Inform., vol. 7, no. 6, pp. 3341–3347, 2024, doi: 10.36040/jati.v7i6.7257.

H. Hidayatullah, Purwantoro, and Y. Umaidah, “Penerapan Naïve Bayes Dengan Optimasi Information Gain Dan Smote Untuk Analisis Sentimen Pengguna Aplikasi Chatgpt,” JATI (Jurnal Mhs. Tek. Inform., vol. 7, no. 3, pp. 1546–1553, 2023, doi: 10.36040/jati.v7i3.6887.

I. S. K. Idris, Y. A. Mustofa, and I. A. Salihi, “Analisis Sentimen Terhadap Penggunaan Aplikasi Shopee Mengunakan Algoritma Support Vector Machine (SVM),” Jambura J. Electr. Electron. Eng., vol. 5, no. 1, pp. 32–35, 2023, doi: 10.37905/jjeee.v5i1.16830.

T. N. Wijaya, R. Indriati, and M. N. Muzaki, “Analisis Sentimen Opini Publik Tentang Undang-Undang Cipta Kerja Pada Twitter,” Jambura J. Electr. Electron. Eng., vol. 3, no. 2, pp. 78–83, 2021, doi: 10.37905/jjeee.v3i2.10885.

M. A. S. Nugroho, D. Susilo, and D. Retnoningsih, “Analisis sentimen ulasan aplikasi ”access by kai” menggunakan algoritma machine learning,” J. TEKINKOM, vol. 7, no. 2, pp. 820–827, 2024, doi: 10.37600/tekinkom.v7i2.1854.

S. Azhari, N. Rahaningsih, R. D. Dana, and Mulyawan, “Peningkatan akurasi analisis sentimen pada aplikasi loklok dengan metode naïve bayes,” JITET (Jurnal Inform. dan Tek. Elektro Ter., vol. 13, no. 1, pp. 1132–1146, 2025, doi: http://dx.doi.org/10.23960/jitet.v12i3.5848 PENINGKATAN.

I. Amelia, Sugiyono, F. M. Sarimole, and Tundo, “Analisis Sentimen Tanggapan Pengguna Media Sosial X Terhadap Program Beasiswa KIP-Kuliah dengan Menggunakan Algoritma Support Vector Machine ( SVM ),” J. Indones. Manaj. Inform. dan Komun., vol. 5, no. 3, pp. 2994–3003, 2024, doi: https://doi.org/10.35870/jimik.v5i3.990.

I. K. Najibulloh, I. Tahyudin, and D. I. S. Saputra, “Analisis Sentimen Ulasan Co-Pilot Google Play dengan SVM , Neural Network , dan Decision Tree,” Edumatic J. Pendidik. Inform., vol. 9, no. 1, pp. 275–283, 2025, doi: 10.29408/edumatic.v9i1.29673.

J. Hermanto, “Klasifikasi Teks Humor Bahasa Indonesia Memanfaatkan SVM,” J. Inf. Syst. Hosp. Technol., vol. 3, no. 01, pp. 39–48, 2021, doi: 10.37823/insight.v3i01.118.

T. Arifqi, N. Suarna, and W. Prihartono, “Penggunaan Naive Bayes Dalam Menganalisis Sentimen Ulasan Aplikasi Mcdonald’s Di Indonesia,” JATI (Jurnal Mhs. Tek. Inform., vol. 8, pp. 1949–1956, Apr. 2024, doi: 10.36040/jati.v8i2.8740.

M. Saputra and S. Wahyuni, “Analisis Sentimen Pengguna Pada Aplikasi Bank Digital Krom Dengan Algoritma Support Vector Machine,” INFOTECH J., vol. 10, no. 2, pp. 327–332, 2024, doi: https://doi.org/10.31949/infotech.v10i2.11801 INFOTECH.

Eskiyaturrofikoh and R. R. Suryono, “Analisis Sentimen Aplikasi X Pada Google Play Store Menggunakan Algoritma Naïve Bayes Dan Support Vector Machine (Svm),” JIPI(Jurnal Ilm. Penelit. dan Pembelajaran Inform., vol. 9, no. 3, pp. 1408–1419, 2024, [Online]. Available: https://www.jurnal.stkippgritulungagung.ac.id/index.php/jipi/article/view/5392