Development of a Chatbot Based on Natural Language Processing and Multinomial Naïve Bayes for Optimising Academic Administrative Services in Higher Education
DOI:
https://doi.org/10.52436/1.jutif.2026.7.4.5443Keywords:
Chatbot, Digital Administration, Multinomial Naive Bayes, Natural Language Processing, User Acceptance TestingAbstract
The transformation of academic administrative services in higher education requires intelligent systems to automate routine information delivery. This study presents a web-based chatbot using Natural Language Processing (NLP) and the Multinomial Naïve Bayes (MNB) algorithm to enhance administrative services. The system classifies user queries into 20 intent categories and provides real-time responses. Training and testing were conducted using labelled text data. The model achieved 93.62% training accuracy and 63.38% test accuracy, indicating potential overfitting due to the limited variety of test data. Evaluation metrics showed 63% precision, 60% recall, and a 61% F1-score, reflecting stable classification performance. User Acceptance Testing (UAT) was conducted with students and administrative staff, showing acceptance rates of 83.5% and 84%, respectively. These results indicate strong usability and relevance of the chatbot in academic contexts. The system integrates text classification, NLP, and user evaluation within a single framework to address repetitive administrative tasks. This research contributes a practical solution tailored to the needs of academic institutions by combining machine learning techniques with direct user evaluation. The chatbot provides faster and more consistent access to academic information, thereby reducing dependency on manual administrative processes. Future work may enhance classification accuracy by utilising richer datasets and alternative algorithms.
Downloads
References
KEMENDIKBURISTEK, Peraturan Mentri Pendidikan, Kebudayaan, Riset, dan Teknologi Republik Indonesia Nomor 53 Tahun 2023 Tentang Penjaminan Mutu Pendidikan Tinggi. Indonesia: Database Peraturan JDIH BPK, 2023. Accessed: Nov. 13, 2025. [Online]. Available: https://peraturan.bpk.go.id/Details/265158/permendikbudriset-no-53-tahun-2023
Y. B. Bhakti, M. R. R. Simorangkir, A. Tjalla, and A. Sutisna, “Kendala Implementasi Kebijakan Merdeka Belakar Kampus Merdeka (MBKM) di Perguruan Tinggi,” Research and Development Journal of Education, vol. 8, no. 2, p. 783, Sep. 2022, doi: 10.30998/rdje.v8i2.12865.
A. Endurance, O. N. Eunice, U. C. Anthonia, A. E. Sebastine, and O. U. Christiana, “Integration of Artificial Intelligence Tool (Ai-Chatbot) into Teaching and Learning: A Panacea for Improving Universities`Educational and Administrative Duties in South-South, Nigeria,” J Comput Sci Syst Biol, vol. 14:06, 2021.
A. Faisal, N. Jhanjhi, H. Ashraf, S. K. Ray, and F. Ashfaq, “A Comprehensive Review of Machine Learning Models: Principles, Applications, and Optimal Model Selection,” Mar. 24, 2025. doi: 10.36227/techrxiv.174285687.71966152/v1.
C. McGrath, A. Farazouli, and T. Cerratto-Pargman, “AI Chatbots in Higher Education. A state-of-the-art review of an emerging research area,” Jan. 26, 2024. doi: 10.21203/rs.3.rs-3893749/v1.
J. Bhat, “Automating Higher Education Administrative Processes with AI-Powered Workflows,” International Journal of Emerging Trends in Computer Science and Information Technology, vol. 4, no. 4, pp. 147–157, Dec. 2023, doi: 10.63282/3050-9246.IJETCSIT-V4I4P116.
M. Furqan, S. Sriani, and M. N. Shidqi, “Chatbot Telegram Menggunakan Natural Language Processing,” Walisongo Journal of Information Technology, vol. 5, no. 1, pp. 15–26, Jun. 2023, doi: 10.21580/wjit.2023.5.1.14793.
N. Mamuriyah et al., “Pembangunan Chatbot Interaktif dengan Menggunakan Algoritma Naive Bayes,” vol. 4, no. 2, pp. 82–94, 2024, doi: 10.51903/informatika.v4i2.864.
A. Aloqayli and H. Abdelhafez, “Intelligent Chatbot for Admission in Higher Education,” International Journal of Information and Education Technology, vol. 13, no. 9, pp. 1348–1357, Sep. 2023, doi: 10.18178/ijiet.2023.13.9.1937.
A. Puspitasari, A. N. Paradhita, Y. W. Tineka, V. Sulistyowati, N. K. S. Noriska, and Haryanto, “Natural Language Processing (NLP) Technology for Chatbot Website,” Jurnal Penelitian Pendidikan IPA, vol. 10, no. SpecialIssue, pp. 319–324, Aug. 2024, doi: 10.29303/jppipa.v10ispecialissue.8241.
K. Mikael, C. Oz, T. A. Rashid, and G. S. Nariman, “A Hybrid Chatbot Model for Enhancing Administrative Support in Education: Comparative Analysis, Integration, and Optimization,” IEEE Access, vol. 13, pp. 50741–50760, 2025, doi: 10.1109/ACCESS.2025.3552501.
H. T. Hien, P. N. Cuong, L. N. H. Nam, H. L. T. K. Nhung, and L. D. Thang, “Intelligent assistants in higher-education environments: The FIT-EBOt, a chatbot for administrative and learning support,” in ACM International Conference Proceeding Series, Association for Computing Machinery, Dec. 2018, pp. 69–76. doi: 10.1145/3287921.3287937.
M. Kamran Abid, S. U. Din, S. Fareed, A. Naeem, M. Fuzail, and M. Khan, “Integrating chatbots in educational administration for improved language learning outcomes,” LGU Research Journal of Computer Science & IT, vol. Vol(7):Issue(4), no. 7, Oct. 2023, doi: 10.54692/lgurjcsit.2023.074494.
C. D. B. Majorana, R. B. Gonçalves, F. L. A. Neto, and R. Z. Camargo, “Enhancing Administrative Efficiency in Higher Education with AI: A Chatbot Solution,” Review of Artificial Intelligence in Education, vol. 3, p. e023, Oct. 2022, doi: 10.37497/rev.artif.intell.educ.v3i00.23.
H. A. Rasheed, J. Zenkert, C. Weber, and M. Fathi, “Conversational Chatbot System for Student Support in Administrative Exam Information,” in Proceedings of ICERI2019 Conference, Seville: IATED, Nov. 2019, pp. 8294–8301. Accessed: Jan. 11, 2026. [Online]. Available: https://library.iated.org/view/ABURASHEED2019CON
E. Ufomba, E. R. Ufomba, E.-O. Judith, and O. C. Daniel, “The Role of Arficial Intelligence in Enhancing Administrave and Learning Efficiency in Higher Educaon: Insights from Abia State University, Uturu,” International Academy Journal of Administration, Education and Society, vol. 6, no. 3, pp. 189–202, Sep. 2024, doi: :2721-3750-3-644.
M. Poriye, P. Mittal, N. Sharma, and U. Delhi, “Organized by HMR Institute of Technology and Management,” 2025. doi: 10.21467/proceedings.7.6.
M. Hasal, J. Nowaková, K. Ahmed Saghair, H. Abdulla, V. Snášel, and L. Ogiela, “Chatbots: Security, privacy, data protection, and social aspects,” Concurr Comput, vol. 33, no. 19, Oct. 2021, doi: 10.1002/cpe.6426.
G. R. S. Silva and E. D. Canedo, “Privacy in Chatbot Conversation-Driven Development: A Comprehensive Review and Requirements Proposal,” ACM Transactions on Software Engineering and Methodology, vol. 34, no. 7, Aug. 2025, doi: 10.1145/3730578.
N. Widaad and D. Anggraini, “SENTIMENT ANALYSIS OF CHATGPT APP USER REVIEWS USING SVM AND CNN METHODS,” Jurnal Teknik Informatika (Jutif), vol. 5, no. 6, pp. 1687–1700, Dec. 2024, doi: 10.52436/1.JUTIF.2024.5.6.4010.
J. H. Cabot and E. G. Ross, “Evaluating prediction model performance,” Surgery (United States), vol. 174, no. 3, pp. 723–726, Sep. 2023, doi: 10.1016/j.surg.2023.05.023.
M. Muliyono and S. Sumijan, “Identifikasi Chatbot dalam Meningkatkan Pelayanan Online Menggunakan Metode Natural Language Processing,” Jurnal Informatika Ekonomi Bisnis, pp. 142–147, Sep. 2021, doi: 10.37034/infeb.v3i4.102.
S. R. Natasia, Y. T. Wiranti, and A. Parastika, “Acceptance analysis of NUADU as e-learning platform using the Technology Acceptance Model (TAM) approach,” in Procedia Computer Science, Elsevier B.V., 2021, pp. 512–520. doi: 10.1016/j.procs.2021.12.168.
M. Barbosa et al., “UX of Chatbots: An Exploratory Study on Acceptance of User Experience Evaluation Methods,” in International Conference on Enterprise Information Systems, ICEIS - Proceedings, Science and Technology Publications, Lda, 2022, pp. 355–363. doi: 10.5220/0011090100003179.
R. Suman and S. Sahibuddin, “User acceptance testing in mobile health applications: An overview and the Challenges,” in ACM International Conference Proceeding Series, Association for Computing Machinery, 2019, pp. 145–149. doi: 10.1145/3322645.3322670.
N. E. Andrews et al., “Acceptability of a Pain History Assessment and Education Chatbot (Dolores) Across Age Groups in Populations With Chronic Pain: Development and Pilot Testing.,” JMIR Form Res, vol. 7, no. 1, p. e47267, Oct. 2023, doi: 10.2196/47267.
T. Willem, A. Wollek, T. Cheslerean-Boghiu, M. Kenney, and A. Buyx, “The Social Construction of Categorical Data: Mixed Methods Approach to Assessing Data Features in Publicly Available Datasets,” JMIR Med Inform, vol. 13, 2025, doi: 10.2196/59452.
J. Homepage, A. Roihan, P. Abas Sunarya, and A. S. Rafika, “Pemanfaatan Machine Learning dalam Berbagai Bidang: Review paper,” IJCIT (Indonesian Journal on Computer and Information Technology), vol. 5 (1), no. 1, pp. 75–82, May 2019.
Z. B. Toyibah, Y. N. Putri, P. Puandini, Z. M. Widodo, and A. T. Ni’mah, “Perbandingan Kinerja Algoritma Multinomial Naïve Bayes dan Logistic Regression pada Analisis Sentimen Movie Ratings IMDB,” Jurnal Ilmiah Edutic : Pendidikan dan Informatika, vol. 10, no. 2, pp. 181–189, Nov. 2024, doi: 10.21107/edutic.v10i2.28150.
A. Muhidin, M. Danny, and E. Rilvani, “Algoritme Multinomial Naïve Bayes Pada Aplikasi Chatbot Layanan Informasi Berbasis Teks,” Jurnal Ilmiah Komputer, vol. Vol 19 No 1, pp. 71–80, Feb. 2023, [Online]. Available: https://www.pelitabangsa.ac.id/’
H. W. Dhany, R. Ramadhan, M. Z. F. Nugraha, D. A. Pratama, M. Irsyad, and T. S. Tambunan, “Penggunaan Chatbot Dalam Memberikan Layanan Bimbingan Mahasiswa Di Brida Kota Medan,” Jurnal Minfo Polgan, vol. 14, no. 1, pp. 1131–1137, Jul. 2025, doi: 10.33395/jmp.v14i1.14992.
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Rahmalia Syahputri, Ammar Ismail Kochan, Dika Tondo Widakdo
This work is licensed under a Creative Commons Attribution 4.0 International License.
</a
