PERFORMANCE COMPARISON OF NAIVE BAYES AND BIDIRECTIONAL LSTM ALGORITHMS IN BSI MOBILE REVIEW SENTIMENT ANALYSIS

Hannatul  Ma'we; Ario Yudo Husodo; Budi Irmawati

doi:10.52436/1.jutif.2024.5.6.4178

Hannatul Ma'we Master of Information Technology, Faculty of Engineering, University of Mataram, Indonesia
Ario Yudo Husodo Master of Information Technology, Faculty of Engineering, University of Mataram, Indonesia
Budi Irmawati Master of Information Technology, Faculty of Engineering, University of Mataram, Indonesia

DOI: https://doi.org/10.52436/1.jutif.2024.5.6.4178

Keywords: sentiment analysis, bidirectional LSTM, BSI mobile, deep learning, machine learning, naive bayes

Abstract

Currently, almost all banks have used mobile banking in conducting banking transactions, one of which is Bank Syariah Indonesia (BSI). BSI mobile is still classified as a new mobile banking application compared to other mobile banking, this certainly still has a low rating and really needs feedback from users which can be seen through reviews on the Google Play Store application. Input in the form of criticism and suggestions from BSI mobile users can be used by BSI mobile as a suggestion for careful supervision and evaluation material in improving its services. This study aims to find the best algorithm to analyze review sentiment on the Google Play Store for the BSI mobile application and provide an overview of the response of application users to application developers based on the results of review data processing. The data mining methodology used in this study is CRISP-DM, using a dataset collected for 6 years (2018-2023) which is annotated into positive and negative labels manually, then modeled using 2 algorithms, namely Naïve Bayes (NB) and Bidirectional LSTM (BiLSTM). The contribution of this study is to test, evaluate and compare the two algorithms (NB and BiLSTM) using the K-Fold Cross Validation (NB) testing model and over-sampling techniques to the minority class (negative) then provide recommendations for the best algorithm. The conclusion of the study is that the BiLSTM algorithm is superior to NB with an accuracy of 94.90 % while the NB algorithm is 94%. In addition, the over-sampling technique is more optimal in increasing the accuracy of the algorithm's performance compared to without over-sampling.

Downloads

Download data is not yet available.

References

R. Adawiyah and J. Nugraha, "Sentiment Analysis on Mobile Banking Application Using Naive Bayes Classifer and Association Methods," Int. J.Eng. Technol. , vol. 7, no. 4.15, p. 244, 2018, doi: 10.14419/ijet.v7i4.15.22998.

H. Cam, AV Cam, U. Demirel, and S. Ahmed, “Sentiment analysis of financial Twitter posts on Twitter with the machine learning classifiers,” Heliyon , vol. 10, no. 1, p. e23784, 2024, doi: 10.1016/j.heliyon.2023.e23784.

M. Eksa Permana, H. Ramadhan, I. Budi, A. Budi Santoso, and P. Kresna Putra, “Sentiment analysis and topic detection of mobile banking application review,” 2020 5th Int. Conf. Informatics Comput. ICIC 2020 , 2020, doi: 10.1109/ICIC50835.2020.9288616.

AA Arifiyanti, NR Shantika, and AO Syafira, “Sentiment Analysis of Bsi Mobile User Reviews on Google Play Using a Supervised Learning Approach,” J. Inform. Polinema , vol. 9, no. 3, pp. 283–288, 2023 , doi: 10.33795/jip.v9i3.1003.

M. V. Kusuma, S. Juanita, and U. B. Luhur, “COMPARISON OF K-NEAREST NEIGHBORS AND NAÏVE BAYES CLASSIFIER ALGORITHMS IN SENTIMENT ANALYSIS OF USER REVIEWS FOR PERBANDINGAN ALGORITMA K-NEAREST NEIGHBHORS DAN NAÏVE BAYES CLASSIFIER PADA ANALISIS SENTIMEN ULASAN PENGGUNA APLIKASI,” vol. 5, no. 5, pp. 1433–1441, 2024.

H. Akbar, D. Aryani, M. K. M. Al-shammari, and M. B. Ulum, “SENTIMENT ANALYSIS FOR E-COMMERCE PRODUCT REVIEWS BASED ON FEATURE FUSION AND BIDIRECTIONAL LONG SHORT-TERM MEMORY,” vol. 5, no. 4, pp. 1385–1391, 2024.

W. L. Roldan-Baluis, N. A. Zapata, and M. S. M. Vásquez, “The Effect of Natural Language Processing on the Analysis of Unstructured Text: A Systematic Review,” Int. J. Adv. Comput. Sci. Appl., vol. 13, no. 5, pp. 43–51, 2022, doi: 10.14569/IJACSA.2022.0130507.

B. Andrian, T. Simanungkalit, I. Budi, and AF Wicaksono, “Sentiment Analysis on Customer Satisfaction of Digital Banking in Indonesia,” Int. J. Adv. Comput. Sci. Appl. , vol. 13, no. 3, pp. 466–473, 2022, doi: 10.14569/IJACSA.2022.0130356.

X. Li, Z. Zhang, and K. Stefanidis, “Sentiment-aware Analysis of Mobile Apps User Reviews Regarding Particular Updates,” J. Icsea , no. January 2020, p. 9, 2018, [Online]. Available: https://people.uta.fi/~kostas.stefanidis/docs/icsea18.pdf

NT Romadloni, I. Santoso, and S. Budilaksono, “Comparison of Naïve Bayes, KNN and Decision Tree Methods on KRL Commuter Line Transportation Sentiment Analysis,” IKRA-ITH Inform. J. Comput. and Inform. , vol. 3, no. 2, pp. 1–9, 2019.

H. Wisnu, M. Afif, and Y. Ruldevyani, “Sentiment analysis on customer satisfaction of digital payment in Indonesia: A comparative study using KNN and Naïve Bayes,” J. Phys. Conf. Ser., vol. 1444, no. 1, 2020, doi: 10.1088/1742-6596/1444/1/012034.

D. A. Nurdeni, I. Budi, and A. B. Santoso, “Sentiment Analysis on Covid19 Vaccines in Indonesia: From the Perspective of Sinovac and Pfizer,” 3rd 2021 East Indones. Conf. Comput. Inf. Technol. EIConCIT 2021, pp. 122–127, 2021, doi: 10.1109/EIConCIT50028.2021.9431852.

G. J. Krishna et al. , “Sentiment Classification of Indian Banks' Customer Complaints,” IEEE Reg. 10 Annu. Int. Conf. Proceedings/TENCON , vol. 2019-Octob, pp. 429–434, 2019, doi: 10.1109/TENCON.2019.8929703.

AD Adhi Putra, “Sentiment Analysis on User Reviews of Bibit and Bareksa Applications with the KNN Algorithm,” JATISI (Journal of Information Technology and Information Systems) , vol. 8, no. 2, pp. 636–646, 2021 , doi: 10.35957/jatisi.v8i2.962.

A. Oktian Permana and Sudin Saepudin, “Comparison of k-nearst neighbor and naïve bayes algorithms on the shopee application,” J. CoSciTech (Computer Sci. Inf. Technol. , vol. 4, no. 1, pp. 25– 32, 2023, doi: 10.37859/coscitech.v4i1.4474.

AI Tanggraeni and MNN Sitokdana, “Sentiment Analysis of E-Government Applications on Google Play Using the Naïve Bayes Algorithm,” JATISI (Journal of Information Technology and Information Systems) , vol. 9, no. 2, pp. 785–795, 2022, doi: 10.35957/jatisi.v9i2.1835.

MS Yupi Kuspandi Putra, “Comparison of Naive Bayes and Naive Bayes Algorithms,” J. Inform. and Technol. , vol. 2, no. 3, pp. 61–69, 2019.

DT Ksatria, Y. Yunefri, and LL Van FC, “Sentiment Analysis of Mypertamina Application User Reviews on Google Playstore Using K-Nearest Neighbor and Naïve Bayes,” Teknol. Inf. , vol. 2, no. 1, pp. 213–227, 2023, [Online]. Available: https://journal.unilak.ac.id/index.php/Semaster/article/view/18526

S. Mulyani and R. Novita, “Implementation of the Naive Bayes Classifier Algorithm for Classification of Community Sentiment About Depression on Youtube,” J. Tek. Inform., vol. 3, no. 5, pp. 1355–1361, 2022, doi: 10.20884/1.jutif.2022.3.5.374.

T. T. Mengistie and D. Kumar, “Deep Learning Based Sentiment Analysis on COVID-19 Public Reviews,” 3rd Int. Conf. Artif. Intell. Inf. Commun. ICAIIC 2021, pp. 444–449, 2021, doi: 10.1109/ICAIIC51459.2021.9415191.

M. Arbane, R. Benlamri, Y. Brik, and A. D. Alahmar, “Social media-based COVID-19 sentiment classification model using Bi-LSTM,” Expert Syst. Appl., vol. 212, no. June 2022, p. 118710, 2023, doi: 10.1016/j.eswa.2022.118710.

U. B. Mahadevaswamy and P. Swathi, “Sentiment Analysis using Bidirectional LSTM Network,” Procedia Comput. Sci., vol. 218, pp. 45–56, 2022, doi: 10.1016/j.procs.2022.12.400.

D. R. Alghifari, M. Edi, and L. Firmansyah, “Implementasi Bidirectional LSTM untuk Analisis Sentimen Terhadap Layanan Grab Indonesia,” J. Manaj. Inform., vol. 12, no. 2, pp. 89–99, 2022, doi: 10.34010/jamika.v12i2.7764.

Y. A. Wubet and K. Lian, “International Journal of Information How can we detect news surrounding community safety crisis incidents in the internet ? Experiments using attention-based Bi-LSTM models,” Int. J. Inf. Manag. Data Insights, vol. 4, no. 1, p. 100227, 2024, doi: 10.1016/j.jjimei.2024.100227.

A. Alzhrani, A. Alatawi, B. Alsharari, and U. Albalawi, “Towards Security Awareness of Mobile Applications using Semantic-based Sentiment Analysis,” Int. J. Adv. Comput. Sci. Appl., vol. 13, no. 4, pp. 800–809, 2022, doi: 10.14569/IJACSA.2022.0130493.

U. Uslu, Ö. D. Incel, and G. I. Alptekin, “Evaluation of Deep Learning Models for Continuous Authentication Using Behavioral Biometrics,” Procedia Comput. Sci., vol. 225, pp. 1272–1281, 2023, doi: 10.1016/j.procs.2023.10.115.

M. J. Islam, R. Datta, and A. Iqbal, “Actual rating calculation of the zoom cloud meetings app using user reviews on google play store with sentiment annotation of BERT and hybridization of RNN and LSTM,” Expert Syst. Appl., vol. 223, no. March, p. 119919, 2023, doi: 10.1016/j.eswa.2023.119919.

P. F. Muhammad, R. Kusumaningrum, and A. Wibowo, “Sentiment Analysis Using Word2vec and Long Short-Term Memory (LSTM) for Indonesian Hotel Reviews,” Procedia Comput. Sci., vol. 179, no. 2020, pp. 728–735, 2021, doi: 10.1016/j.procs.2021.01.061.

M. S. Islam and E. Hossain, “Foreign exchange currency rate prediction using a GRU-LSTM hybrid network,” Soft Comput. Lett., vol. 3, no. June 2020, p. 100009, 2021, doi: 10.1016/j.socl.2020.100009.

B. Dangsawang and S. Nuchitprasitchai, “A machine learning approach for detecting customs fraud through unstructured data analysis in social media,” Decis. Anal. J., vol. 10, no. February, p. 100408, 2024, doi: 10.1016/j.dajour.2024.100408.

Z. H. Kilimci, H. Yoruk, and S. Akyokus, “Sentiment Analysis Based Churn Prediction in Mobile Games using Word Embedding Models and Deep Learning Algorithms,” INISTA 2020 - 2020 Int. Conf. Innov. Intell. Syst. Appl. Proc., 2020, doi: 10.1109/INISTA49547.2020.9194624.