Enhancing Question Classification in Educational Chatbots Using RASA Natural Language Understanding

Zaenur Dwi Christanto; Kristophorus Hadiono

doi:10.52436/1.jutif.2026.7.1.4732

Authors

Zaenur Dwi Christanto Graduate School of Information Technology, Faculty of Information Technology and Industries, Stikubank Semarang University, Indonesia
Kristophorus Hadiono Graduate School of Information Technology, Faculty of Information Technology and Industries, Stikubank Semarang University, Indonesia

DOI:

https://doi.org/10.52436/1.jutif.2026.7.1.4732

Keywords:

Artificial Intelligence, Chatbot, Informatics Learning, Rasa Framework, Student Understanding

Abstract

This research develops a chatbot model based on Rasa Framework to understand and respond to questions related to informatics learning, addressing the critical need for personalized AI-driven educational tools in Indonesian secondary education. The model is trained to recognize various patterns of student questions about informatics materials, especially the topic of number conversion. Using Natural Language Understanding (NLU), the chatbot model is developed to process natural language and classify the intent of student questions. Evaluation of the model using the confusion matrix showed good performance with 91.5% accuracy, 94.4% average precision, and 100% recall. The test results showed that the model was able to correctly classify various types of intent, where eight out of nine intents achieved a perfect precision of 100%, with one intent, tutorial_calculation_octal_to_decimal, having a precision of 50%. The 100% recall across all intents demonstrates the model's comprehensive ability to identify all cases requiring responses, ensuring no student queries are missed. This research significantly contributes to computer science education by validating RASA's effectiveness for domain-specific NLU in low-resource educational settings, providing a scalable foundation for AI-based learning assistance tools that can enhance digital literacy and computational thinking skills among junior high school students.

Downloads

Download data is not yet available.

References

A. M. Sayaf, M. M. Alamri, M. A. Alqahtani, and W. M. Al-Rahmi, “Information and Communications Technology Used in Higher Education: An Empirical Study on Digital Learning as Sustainability,” Sustainability, vol. 13, no. 13, p. 7074, Jun. 2021, doi: 10.3390/su13137074.

A. Szymkowiak, B. Melović, M. Dabić, K. Jeganathan, and G. S. Kundi, “Information technology and Gen Z: The role of teachers, the internet, and technology in the education of young people,” Technol. Soc., vol. 65, p. 101565, May 2021, doi: 10.1016/j.techsoc.2021.101565.

S. Shanta and J. G. Wells, “T/E design based learning: assessing student critical thinking and problem solving abilities,” Int. J. Technol. Des. Educ., vol. 32, no. 1, pp. 267–285, Mar. 2022, doi: 10.1007/s10798-020-09608-8.

S. Pratasik and B. M. Ahyar, “Pengembangan Media Pembelajaran Pada Mata Pelajaran Informatika MTS,” Edutik J. Pendidik. Teknol. Inf. dan Komun., vol. 2, no. 3, pp. 359–373, Jun. 2022, doi: 10.53682/edutik.v2i3.5282.

C. Zhang, I. Khan, V. Dagar, A. Saeed, and M. W. Zafar, “Environmental impact of information and communication technology: Unveiling the role of education in developing countries,” Technol. Forecast. Soc. Change, vol. 178, p. 121570, May 2022, doi: 10.1016/j.techfore.2022.121570.

C. W. Okonkwo and A. Ade-Ibijola, “Chatbots applications in education: A systematic review,” Comput. Educ. Artif. Intell., vol. 2, p. 100033, 2021, doi: 10.1016/j.caeai.2021.100033.

M. Y. Uohara, J. N. Weinstein, and D. C. Rhew, “The Essential Role of Technology in the Public Health Battle against COVID-19,” Popul. Health Manag., vol. 23, no. 5, pp. 361–367, 2020, doi: 10.1089/pop.2020.0187.

D. G. S. Ruindungan and A. Jacobus, “Chatbot Development for an Interactive Academic Information Services using the Rasa Open Source Framework,” J. Tek. Elektro dan Komput., vol. 10, no. p-ISSN : 2301-8402, e-ISSN : 2685-368X ,available at: https://ejournal.unsrat.ac.id/index.php/elekdankom, pp. 61–68, 2021.

Rakesh Kumar Sharma, “An Analytical Study and Review of open source Chatbot framework, Rasa,” Int. J. Eng. Res., vol. V9, no. 06, Jun. 2020, doi: 10.17577/IJERTV9IS060723.

J. Doshi, “Chatbot User Interface for Customer Relationship Management using NLP models,” in 2021 International Conference on Artificial Intelligence and Machine Vision (AIMV), Sep. 2021, pp. 1–4, doi: 10.1109/AIMV53313.2021.9670914.

X. H. Qin et al., “Using a one-dimensional convolutional neural network with a conditional generative adversarial network to classify plant electrical signals,” Comput. Electron. Agric., vol. 174, no. April, p. 105464, 2020, doi: 10.1016/j.compag.2020.105464.

A. Rachman, I. Mardhiyah, and M. Jannah, “Implementasi Chatbot FAQ pada Aplikasi Monev Kinerja Direktorat Jenderal Anggaran Menggunakan Framework Rasa Open Source,” J. KLIK Kaji. Ilm. Inform. dan Komput., vol. 4, no. 1, pp. 62–72, 2023, doi: 10.30865/klik.v4i1.1020.

L. Anindyati, “Analisis dan Perancangan Aplikasi Chatbot Menggunakan Framework Rasa dan Sistem Informasi Pemeliharaan Aplikasi (Studi Kasus: Chatbot Penerimaan Mahasiswa Baru Politeknik Astra),” J. Teknol. Inf. dan Ilmu Komput., vol. 10, no. 2, pp. 291–300, Apr. 2023, doi: 10.25126/jtiik.20231026409.

M. C. Wijanto et al., “Informatika untuk SMP Kelas VII,” in Pusat Kurikulum dan Perbukuan, Jakarta Pusat: Pusat Kurikulum dan Perbukuan, Badan Penelitian dan Pengembangan dan Perbukuan, Kemdikbudristek, 2021.

S. HV and S. S, “Implementation of an Educational Chatbot using Rasa Framework,” Int. J. Innov. Technol. Explor. Eng., vol. 11, no. 9, pp. 29–35, Aug. 2022, doi: 10.35940/ijitee.G9189.0811922.

T. Maulida et al., “Visualization of Front-End Data Logger Internet of Things Technology using Vue.Js Framework,” in 2022 6th International Conference on Information Technology, Information Systems and Electrical Engineering (ICITISEE), Dec. 2022, pp. 693–698, doi: 10.1109/ICITISEE57756.2022.10057919.

A. N. A. Zumaroh et al., “Development of Application Programming Interface (Api) for Amikom Purwokerto Handsanitizer (Ampuh) Data Logger Visualization,” J. Tek. Inform., vol. 3, no. 3, pp. 791–796, 2022, [Online]. Available: http://jutif.if.unsoed.ac.id/index.php/jurnal/article/view/222.

V. S. Ginting, K. Kusrini, and E. T. Luthfi, “Penerapan Algoritma C4.5 Dalam Memprediksi Keterlambatan Pembayaran Uang Sekolah Menggunakan Python,” J. Teknol. Inf., vol. 4, no. 1, pp. 1–6, 2020, doi: 10.36294/jurti.v4i1.1101.

S. Napi’ah, T. H. Saragih, D. T. Nugrahadi, D. Kartini, and F. Abadi, “Implementation of Monarch Butterfly Optimization for Feature Selection in Coronary Artery Disease Classification Using Gradient Boosting Decision Tree,” J. Electron. Electromed. Eng. Med. Informatics, vol. 5, no. 4, Oct. 2023, doi: 10.35882/jeeemi.v5i4.331.

A. Özdemir, K. Polat, and A. Alhudhaif, “Classification of imbalanced hyperspectral images using SMOTE-based deep learning methods,” Expert Syst. Appl., vol. 178, no. April, p. 114986, Sep. 2021, doi: 10.1016/j.eswa.2021.114986.