Improving RoBERTa Performance through Hyperparameter Optimization for Sentiment Analysis of Indonesian Tourism Reviews

Imamah Imamah; Myo Thida; Fika Hastarita  Rachman; Budi Dwi  Satoto; Sri Herawati; Yeni Kustiyahningsih; Eka Mala Sari  Rochman; Meita Lailatuz  Zakiyah

doi:10.52436/1.jutif.2026.7.3.5672

Authors

Imamah Department of Information Systems, University of Trunojoyo Madura, Bangkalan, Indonesia
Myo Thida Department of Computer Science, University of Illinois, Chicago, USA
Fika Hastarita Rachman Department of Informatics, University of Trunojoyo Madura, Bangkalan, Indonesia
Budi Dwi Satoto Department of Information Systems, University of Trunojoyo Madura, Bangkalan, Indonesia
Sri Herawati Department of Information Systems, University of Trunojoyo Madura, Bangkalan, Indonesia
Yeni Kustiyahningsih Department of Information Systems, University of Trunojoyo Madura, Bangkalan, Indonesia
Eka Mala Sari Rochman Department of Informatics, University of Trunojoyo Madura, Bangkalan, Indonesia
Meita Lailatuz Zakiyah Department of Information Systems, University of Trunojoyo Madura, Bangkalan, Indonesia

DOI:

https://doi.org/10.52436/1.jutif.2026.7.3.5672

Keywords:

Classification, Deep Learning, Sentiment Analysis, Text Mining, Touris Reviews

Abstract

The performance of transformer models such as RoBERTa in sentiment classification is influenced by hyperparameter settings, especially the epoch and batch sizes. However, no previous study has examined the impact of changes in the number of epochs and batch sizes on the performance of each class in classification tasks, especially in Indonesian-language sentiment analysis of tourism reviews. Therefore, this study aims to fill this gap by analyzing the performance of RoBERTa and the impact of various hyperparameter settings on sentiment for each class. The dataset consists of 3,875 reviews from visitors to Lake Sarangan on Google Maps. The batch sizes used in this study are 8 and 16, and the epoch range is 2 to 4. There are three classes of sentiment: negative, neutral, and positive. The results demonstrate that increasing the batch size from 8 to 16 does not linearly improve model performance. The optimal combination of epoch=4 and batch size=8 achieved 91% accuracy, with significant improvements in recall and F1-score across all classes, especially in positive sentiment classification. This research offers valuable insights into fine-tuning RoBERTa for sentiment analysis in Indonesian contexts, providing recommendations for future sentiment analysis tasks in natural language processing.

Downloads

Download data is not yet available.

References

P. Sharma, P. Tomar, and D. Mukherjee, “Sentiment Analysis on Amazon Dataset using Transfer Learning,” in 2022 International Conference on Fourth Industrial Revolution Based Technology and Practices (ICFIRTP), 2022, pp. 160–165, doi: 10.1109/ICFIRTP56122.2022.10059413.

I. Imamah, H. Husni, E. M. Rohman, I. O. Suzanti, and F. A. Mufarroha, “Text mining and Support Vector Machine for Sentiment Analysis of tourist Reviews in Bangkalan Regency,” J. Phys. Conf. Ser., vol. 1477, no. 2, pp. 0–6, 2020, doi: 10.1088/1742-6596/1477/2/022023.

R. Qasim, W. H. Bangyal, M. A. Alqarni, and A. Ali Almazroi, “A Fine-Tuned BERT-Based Transfer Learning Approach for Text Classification,” J. Healthc. Eng., vol. 2022, 2022, doi: 10.1155/2022/3498123.

D. T. Putra and E. B. Setiawan, “Sentiment Analysis on Social Media with Glove Using Combination CNN and RoBERTa,” J. RESTI, vol. 7, no. 3, pp. 457–563, 2023, doi: 10.29207/resti.v7i3.4892.

W. Suwarningsih, R. A. Pratama, F. Y. Rahadika, and M. H. A. Purnomo, “RoBERTa: language modelling in building Indonesian question-answering systems,” Telkomnika (Telecommunication Comput. Electron. Control., vol. 20, no. 6, pp. 1248–1255, 2022, doi: 10.12928/TELKOMNIKA.v20i6.24248.

Y. Liu et al., “Roberta: A robustly optimized bert pretraining approach,” arXiv Prepr. arXiv1907.11692, 2019.

U. K. Immanuel, “Sentiment Analysis of Public Opinions Regarding Ideas of Presidential Candidates in YouTube Video Comments with Robustly Optimized BERT Pretraining Approach,” 2024.

N. A. Semary, W. Ahmed, K. Amin, P. Pławiak, and M. Hammad, “Improving sentiment classification using a RoBERTa-based hybrid model,” Front. Hum. Neurosci., vol. 17, pp. 1–10, 2023, doi: 10.3389/fnhum.2023.1292010.

A. A. Azhari, Y. Sibaroni, and S. S. Prasetiyowati, “Detection of Indonesian Hate Speech in the Comments Column of Indonesian Artists’ Instagram Using the RoBERTa Method,” JIPI, vol. 8, no. 3, pp. 764–773, 2023, doi: 10.29100/jipi.v8i3.3898.

M. Usman et al., “Fine-Tuned RoBERTa Model for Bug Detection in Mobile Games: A Comprehensive Approach,” Computers, vol. 14, no. 4, p. 113, 2025.

D. M. Rathod, K. Patel, A. J. Goswami, S. Degadwala, and D. Vyas, “Exploring Drug Sentiment Analysis with Machine Learning Techniques,” in 2023 International Conference on Inventive Computation Technologies (ICICT), 2023, pp. 9–12, doi: 10.1109/ICICT57646.2023.10134055.

C. P. Chai, “Comparison of text preprocessing methods,” Nat. Lang. Eng., vol. 29, no. 3, pp. 509–553, 2023.

M. Pfeifer and V. P. Marohl, “CentralBankRoBERTa: A fine-tuned large language model for central bank communications,” J. Financ. Data Sci., vol. 9, p. 100114, 2023.

F. I. Kurniadi, N. L. P. S. P. Paramita, E. F. A. Sihotang, M. S. Anggreainy, and R. Zhang, “BERT and RoBERTa Models for Enhanced Detection of Depression in Social Media Text,” Procedia Comput. Sci., vol. 245, pp. 202–209, 2024, doi: https://doi.org/10.1016/j.procs.2024.10.244.

Z. Huang, T. Ban, and Y. Zhang, “A novel approach for malicious URL detection using RoBERTa and sparse autoencoder,” J. Inf. Secur. Appl., vol. 94, no. September, p. 104214, 2025, doi: 10.1016/j.jisa.2025.104214.

A. Jabbary, R. Boostani, F. A. Alenizi, and A. Salih, “RoBERTa , ResNeXt and BiLSTM with self-attention : The ultimate trio for customer sentiment analysis,” Appl. Soft Comput., vol. 164, no. November 2023, p. 112018, 2024, doi: 10.1016/j.asoc.2024.112018.

R. Mohawesh, H. Bany, Y. Jararweh, and M. Alkhalaileh, “Fake review detection using transformer-based enhanced LSTM and RoBERTa,” Int. J. Cogn. Comput. Eng., vol. 5, no. June, pp. 250–258, 2024, doi: 10.1016/j.ijcce.2024.06.001.

M. A. A. Yani and W. Maharani, “Analyzing cyberbullying negative content on twitter social media with the RoBERTa method,” JINAV J. Inf. Vis., vol. 4, no. 1, pp. 61–69, 2023.

L. Yang, J. Wang, and W. Qiu, “RoBERTa-based Multi-Feature Integrated BiLSTM and CNN Model for Ceramic Review Analysis,” IEEE Access, 2025.

O. Ozyegen et al., “Classifying multi-level product categories using dynamic masking and transformer models,” J. Data, Inf. Manag., vol. 4, no. 1, pp. 71–85, 2022.

M. Straka, J. Náplava, J. Straková, and D. Samuel, “RobeCzech: Czech RoBERTa, a monolingual contextualized language representation model,” in International conference on text, speech, and dialogue, 2021, pp. 197–209.

X. D. J. Nguyen and Y. A. Liu, “Methodology for hyperparameter tuning of deep neural networks for efficient and accurate molecular property prediction,” Comput. & Chem. Eng., vol. 193, p. 108928, 2025.

X. Piao, D. Synn, J. Park, and J. K. Kim, “Enabling Large Batch Size Training for DNN Models Beyond the Memory Limit While Maintaining Performance,” IEEE Access, vol. 11, no. September, pp. 102981–102990, 2023, doi: 10.1109/ACCESS.2023.3312572.

J.-Y. Ong, L.-Y. Ong, and M.-C. Leow, “Addressing overfitting in comparative study for deep learning-based classification,” TELKOMNIKA (Telecommunication Comput. Electron. Control., vol. 23, no. 3, pp. 673–681, 2025.

I. Gambo, R. Massenon, R. Oluwaseun, S. Agarwal, and W. Pak, “Identifying and resolving conflict in mobile application features through contradictory feedback analysis,” Heliyon, vol. 10, no. 17, p. e36729, 2024, doi: 10.1016/j.heliyon.2024.e36729.

U. P. Sanjaya et al., “XGBoost for Educational Performance: Comparing SMOTE and SMOTE-TOMEK on Imbalanced Data,” ICCMS (Proceeding Int. Collab. Conf. Multidiscip. Sci., vol. 2, no. 2, pp. 271–281, 2024.

S. Wang, “Development of an automated transformer-based text analysis framework for monitoring fire door defects in buildings,” Sci. Rep., vol. 15, no. 1, pp. 1–22, 2025, doi: 10.1038/s41598-025-27648-9.

N. Bölücü, M. Rybinski, X. Dai, and S. Wan, “An adaptive approach to noisy annotations in scientific information extraction,” Inf. Process. Manag., vol. 61, no. 6, p. 103857, 2024, doi: 10.1016/j.ipm.2024.103857.