IMPROVEMENT OF NAIVE BAYES ALGORITHM IN SENTIMENT ANALYSIS OF SHOPEE APPLICATION REVIEWS ON GOOGLE PLAY STORE

Reviews of the shopee application on the google play store are included in useful information if processed properly. Old or new users can analyze app reviews to get information that can be used to evaluate services. The activity of analyzing application reviews is not enough just to see the number of stars, it is necessary to see the entire contents of the review comments to be able to know the intent of the review. A sentiment analysis system is a system used to automatically analyze a review to obtain information including sentiment information that is part of an online review. The data is classified using Naive Bayes. A total of 1,000 shopee app user reviews were collected to form the sample dataset. The purpose of this study is to determine the sentiment analysis of shopee application reviews in the Google Play Store using the Naive Bayes algorithm. The stages of this research include, data collection, labeling, pre-processing, sentiment classification, and evaluation. In the pre-processing stage there are 6 stages, namely Cleaning, Case folding, Word Normalizer, Tokenizing, Stopword Removal and Stemming. TF-IDF (Term Frequency - Inverse Document Frequency) method is used for word weighting. The data will be grouped into two categories, namely negative and positive. The data will then be evaluated using accuracy parameter testing. The test results show an accuracy value of 81%, this result shows that shopee application reviews tend to be negative.


INTRODUCTION
The rapid development of E-commerce in Indonesia has given rise to many marketplaces that offer various products and services to the public.This phenomenon not only reflects the rapid growth of technology, but also creates various opportunities for consumers or businesses.With the increasing number of marketplace options, consumers can easily search and compare products and prices [1].It also creates healthy competition among marketplaces, encouraging innovation in terms of user experience, services, and promotions.
Currently, the Google Play Store offers a plethora of marketplace applications.Google play store is one of the digital content services owned by Google which contains digital products such as applications, music or songs, books, games and cloud-based media players [2].In the play store there are many features such as rating and review features where users of products from the play store can provide their opinions on the products they have used.
One of the e-commerce applications found on the play store is the shopee application launched in 2015, based on data from (SimilarWeb, May 2023) shopee users in Indonesia are around 161 million people.Because everyone has different opinions and thoughts, there are many opinions that are pro and contra to this application.User reviews are very diverse ranging from good and bad talk, user criticism and there are also those who provide suggestions regarding features.Examining reviews can be simplified by observing the star ratings provided by users.However, these star ratings alone do not provide a comprehensive understanding of the complete review content [3].Review analysis can be done manually by looking at reviews one by one, but if there are too many reviews then we can't do it manually there is a faster and more efficient way to use automated systems such as sentiment analysis [4].
Sentiment analysis is the process of identifying, extracting, and evaluating sentiments or opinions expressed in text or data, particularly in the context of reviews or opinions.The goal of sentiment analysis is to understand people's viewpoints, feelings, or attitudes towards a specific topic or entity.Sentiment analysis can be done manually by humans or automatically using computational algorithms and techniques, such as natural language processing and machine learning.In practical applications, sentiment analysis is often used in customer surveys, social media monitoring, product review analysis, and public feedback analysis to gain insights into perceptions and opinions.This research was conducted because there are so many reviews given by shopee application users but these reviews are not utilized as well as possible.Even the review is only limited to a review without any action, this causes the shopee application rating to drop on PlayStore.
In this research, we will conduct sentiment analysis on reviews of the Shopee application accessible on the Google Play Store.The goal is to evaluate Shopee app users by utilizing the Naive Bayes technique with TF-IDF weighting to classify them into negative and positive categories.Furthermore, this study seeks to assess the precision of the Naive Bayes algorithm in performing sentiment analysis of Shopee application reviews.
Furthermore, as for previous research related to this research, it will be used as the author's research material and used as a reference.In the following, research that discusses sentiment analysis in reviews.
First research was carried out by arif rahman, Ema Utami, Sudarmawan come Amikom University Yogyakarta in 2021 entitled "Analysis of Sentiment on Google Play Store Apps utilizing Naive Bayes Algorithm and Genetic Algorithm" aims for determining level accuracy between the two algorithms then for the dataset used comes from several applications such as shopee, ruang guru, pedia Shopee and gojek.In this study the pedia store dataset have the highest accuracy rate of 96.87 percent, then for Shopee obtained an overall accuracy 96.53 percent, Ruangguru obtained the 95.54%, and Gojek obtained an accuracy of 96.54%.The research collected datasets using the web scrapper application from Google Chrome then the labeling was done manually in the Ms Excel application [5].
Second research was conducted by dedi darwis, nery siskawati, zaenal abidin from teknokrat "In 2021, the University of Indonesia conducted research titled 'Utilization of the Naive Bayes Algorithm for Analyzing Sentiments in National BMKG Twitter Data".Python 3.74 was employed as the primary programming tool, and a dataset comprising 1179 tweets was used for testing.The data was divided into two categories: training data and testing data, with a ratio of 70:825.It's worth noting that the dataset was imbalanced, with more positive reviews than negative ones.Notably, the author opted not to perform feature normalization, resulting in relatively low accuracy test results when applying the Naive Bayes method for classification namely 69.97% [6] .
Third research was conducted by styawati, nirwana hendrastuty, auliya Rahman isnain, ari yanti rahmadhani from teknokrat university of Indonesia in 2021 entitled "Analysis of Public Sentiment Towards the Pre-Employment Card Program on Twitter with the Support Vector Machine Method".Aims to analyze public opinion with data obtained on twitter social media using Support Vector Machine (SVM).Then to measure the performance of SVM classification, the Confusion Matrix method is used.
In this study, two kernels were compared, namely linear and RBF kernels.The evaluation results show that the precision of the linear kernel is 98.67%, the precision is 98%, the recall is 99%, and the F1 score reaches 98%.Meanwhile, the precision of the RBF kernel is 97%, the recall is 98.67%, and the F1 score is also 98%.The accuracy of the RBF kernel reached 98.34%.From these results, it can be concluded that the public opinion of Twitter users towards the preemployment card program during the pandemic tends to be neutral, with a neutrality rate of 98.34%.When viewed from the accuracy aspect, the linear kernel is more accurate than the RBF kernel with an accuracy of 98.67% and 98.34% respectively [7].
Fourth studies was carried out with the aid of Billy, Helen, Enda Esyudha retrieved tanjungpura university in 2018, discussing Indonesian online product opinions to achieve records such as sentiment records that's a part of the web evaluation.The information is run through a classification process the use of the Naive Bayes approach.The sentiment analysis gadget is going via 5 levels, namely records series, pre-processing, phrase weighting, model building, and sentiments class.within the phrase weighting degree, the TF-IDF (term Frequency-Inverse document Frequency) approach usage.Facts is assessed into five lessons: strongly bad, terrible, neutral, tremendous, and strongly advantageous.moreover, the statistics might be evaluated the usage of a confusion matrix by using measuring accuracy, bear in mind, and precision.The take a look at outcomes show that in the 3-magnificence take a look at (bad, neutral, and superb) the first-rate performance is completed with ninety% education facts and 10% test data, ensuing in an accuracy of 78%, bear in mind of 93.33%, and precision of 77%.inside the 5-class take a look at, the high-quality performance turned into also finished with 90% schooling statistics and 10% take a look at information, with accuracy reaching 59.33%, remember 58.33%, and precision 59.33% [8].
The fifth study conducted by Apriani & Gustian in 2019 aims to evaluate the performance of accuracy, class recall, and AUC of sentiment analysis using the Tokopedia application by applying the Naïve Bayes algorithm.The results of this study show that the precision value for class 1 is 1.On the other hand, in class recall, the result is 95.49% (positive class: negative class) and the AUC value is 0.980.From the analyzed reviews, it can be seen that comments with negative sentiments have a percentage of 63.53%, while comments with positive sentiments are only about 36.37% [9].
The cryptocurrency.They proposed a CGWELSTM (Convolutional Gated Word Embedding Long Short-Term Memory) architecture that combines a convolutional layer, bidirectional layer, group enhancement mechanism, attention mechanism, and fully connected layer.The accuracy results of this study show that the CGWELSTM architecture achieves the highest level of accuracy in cryptocurrency-related sentiment analysis.In the dataset used, CGWELSTM achieved a classification accuracy of 85.4001% [10].
Based on this background, author does not conduct the same research as others.This research uses review data from the Shopee application on the Google Play Store, then the method used is the naive bayes algorithm.Later the review data that has been collected will be grouped into negative and positive reviews / comments before a process called sentiment analysis is carried out.

RESEARCH METHODS
Then for the research strategy that will be carried out, explain in Figure 1.The following is an explanation of figure 1 concept study.

Data Selection
In this preliminary stage, the writer conducts information collection and statistics labeling.records taken from the shopee utility on the google play shop using the scraping technique by using using python programming.

Pre-Processing
A pre-processing step in sentiment analysis is the initial stage of a sentiment analysis process.In processing unstructured input data into structured data before the main process such as classification or sentiment analysis is carried out [11].At this stage, several stages will be carried out including the following: 1. Cleaning is the method of cleansing emoticons and emblems executed in this observe.Emoticons and emblems in the evaluation are unnoticed due to the fact These studies most simply make a specialization of the text embodied inside the evaluation.The characters that were ignored included "~", " `", " !", " $", " %", " ^", " &", " *", " (", " )", " _", "-", "+", "=", ":", "'", "comma", "duration", "?".reviews included characters similar to " tolong min, untuk jasa kirim diperbaiki " after processing grow to be " tolong min untuk jasa kirim diperbaiki ". 2. Case folding is a degree that involves changing the text in the file to a standardized shape, that's lowercase.for instance, the comment "Tolong perbaiki fitur pembayaran" will become "tolong perbaiki fitur pembayaran ", the uppercase "T" is converted to lowercase "t".

Word Normalizer is a technique used to rectify
words in reviews in order to generate sentences that adhere to proper and accurate Indonesian grammar.This enhancement is essential to facilitate comprehension for readers regarding the sentence's intended meaning.As an illustration, if there is a comment "knpa lama proses update", after going through the Word Normalization process, it will become "kenapa lama proses update".The change of the word "knpa" to "kenapa" is done to make the message easier to understand. 4. Stopwoard removal is a level of putting off phrases based at the phrases contained inside the conjunction list.phrases inclusive of "di", "dan", "yang", might be removed.5. Tokenizing is a technique used to break the text into phrases, deliberating punctuation and space obstacles.as an instance, the sentence "shopee sekarang lemot" is then transformed into "shopee", "sekarang", "lemot".6. Stemming is the manner of reducing phrases to their simple form or "root" phrase.as an example, the word "batalkan" might be changed to "batal".Stemming is utilized in textual content processing and textual content analysis to help reduce phrase versions and improve consistency in subsequentanalysis or processing.

Text Transformation
This stage of text transformation or attribute formation refers to the process of obtaining the required representation.At this stage the author performs feature extraction with TF-IDF.Here is the formula of TF -IDF.

Data Mining
Data mining is the step-by-step process of extracting relevant, hidden, and potentially valuable information from large and complex data sets [12].The main purpose of data mining is to uncover new knowledge and information that can provide added value or competitive advantage in a domain or industry.This research uses one of the existing algorithms in data mining, namely naive baiyes.
Naive baiyes is a working method for the most popular classifiers with a good average accurization rate.Naive Baiyes classification technique is based on simple probabilities and is designed to be used with the assumption of independence between explanatory variables.This algorithm emphasizes probability learning, here is the formula of naive baiyes [13].
This formula explains that X is evidence, H the hypothesis, then P(H | X) is part of the probability that hypothesis H is true if accompanied by evidence X, or vice versa, P(H | X) enters the probability of hypothesis H in the presence of condition X. P(X | H) can be interpreted as the probability of evidence X in the attendance of hypothesis H, then P(H) is the initial ability of hypothesis H, while P(X) is the initial probability of evidence X..

Data Selection
1. Data Scraping refers to the process of extracting information or data from a source or website.This is done by using algorithms or specialized software that automatically extracts data from web pages and stores it in a format that can be further used and analyzed.Data scraping methods can be done manually, where a person manually copies and pastes information from a website into a file or database, or automatically using specialized software or scripts designed to automatically retrieve data from web pages [14], In this research, scraping shopee application review data is done automatically using the API retrieved from Google Play Store.The results of this data collection are 1000 data, then the data that has been taken is stored in a csv format file.2. Data labeling, also known as data tagging, involves assigning a specific category or label to each data point or sample in a data set.The purpose of data labeling is to give context or meaning to the data so that it can be used to train or test machine learning models [15].In this research, we divided them into negative and positive labels and the subsequent labelin process is done automatically using the python programming language.

Pre-Processing
Derived from the completed labeling process, the subsequent section presents the outcomes of the pre-processing phase.

Text Transformation
The text transformation or attribute generation stage is a step in the process of obtaining the required document representation [16].In this stage, the author performs feature extraction using the TF-IDF method, which is a technique for generating attributes from documents.This process aims to transform the document text into a numerical representation that can be used for further analysis.

Data Mining
After all steps completed, calculations will be made to see accuracy rate using naive baiyes algorithmic approach.The accuracy rate when performing sentiment analysis using this method is quite good with an accuracy value of 81%, then there are also results of precision 80%, recall 97%, f1-score 88% of negative comment data and precision 86%, recall 45%, f1-score 60% of positive comment data.

Frequency of Word Appearance
Frequently occurring words in user reviews can provide information about the app.This time the author will provide an overview of user review based on positive and negative reviews.Visible in Figure 8 and Figure 9.

DISCUSSION
Of the findings obtained from the research that has been carried out, namely sentiment analysis using the naive baiyes algorithm, several things can be used as evaluation material, among others, In this study, the Naive Baiyes algorithm proved to be an accurate algorithm because it produced an accuracy value of 81%.The advantage of this research is that it has good accuracy, precision and recall values.Then the calculation of precision, recall and f1-score is also carried out based on positive and negative reviews.Then there is previous research conducting sentiment analysis with several algorithms where the result is the naive baiyes algorithm which has the highest algorithm until the accuracy rate reaches 65,26 % [17].Therefore, it can be concluded that naive baiyes is accurate enough to be used.The drawback in this study is that it uses a fairly small dataset so that the results are less satisfying, for further researchers can add the number of dataset.

CONCLUSION
The research findings indicate that an accuracy rate of 81% was attained.Additionally, there were precision results of 80% and recall of 97%, resulting in an f1-score of 88% for negative comments.For positive comments, the precision was 86%, recall was 45 percent, and f1-score was 60%.The survey employed The TF-IDF combined with the Naive Bayes algorithm weighting technique to categorize user reviews into either negative or positive sentiments.Furthermore, the sentiment analysis of Shopee app users reveals a prevailing negative sentiment towards Shopee's services.This suggests that there are areas where Shopee could improve, particularly in terms of features and other aspects.Based on the analysis, it can be concluded that the data sourced from the Google Play Store platform for the Shopee application indicates a higher frequency of negative reviews compared to positive ones.The negative reviews often revolve around issues with the application's features, shipping process, and frequent slow updates.Conversely, positive feedback frequently highlights various promotions offered by Shopee, such as free shipping, product discounts, and cashback.

Figure 2 .
Figure 2. Script for scraping data

Figure 4 .
Figure 4. Results of labeling negative and positive reviews.

Figure 7 .
Figure 7. Calculation results with the naive baiyes method.