Optimizing Multimodal Health Chatbots through the Integration of Medical Text and Images

Authors

  • Raditya Danar Dana Informatics Management, STMIK IKMI Cirebon, Indonesia
  • Mulyawan Information System, STMIK IKMI Cirebon, Indonesia
  • Agus Bahtiar Information System, STMIK IKMI Cirebon, Indonesia
  • Odi Nurdiawan Informatics Management, STMIK IKMI Cirebon, Indonesia

DOI:

https://doi.org/10.52436/1.jutif.2026.7.2.5328

Keywords:

Adaptive augmentation, CNN, Digital image, Model robustness, Resolution

Abstract

This study is motivated by the growing need for image-classification systems that remain accurate despite variations in image quality commonly found in real-world environments. Differences in image resolution often lead to decreased performance of Convolutional Neural Network (CNN) models, particularly in scenarios involving limited acquisition devices. This research aims to analyze the effect of image-resolution variations on CNN robustness by applying an adaptive augmentation strategy. An experimental approach was employed by manipulating independent variables namely image-resolution levels and augmentation techniques and observing their impact on accuracy, validation stability, and model generalization. The results show that medium-resolution images (128×128 px) combined with adaptive augmentation produce the best performance, yielding the highest validation accuracy and reduced overfitting compared to other configurations. The urgency of this study lies in its practical contribution to developing efficient image-classification models suitable for resource-constrained environments. Scientifically, the findings provide a structured mapping of the relationship between resolution, augmentation, and model stability, offering a foundation for designing more robust CNN architectures adaptable to real-world data variability.

Downloads

Download data is not yet available.

References

G. V. Research, “Artificial intelligence in healthcare market size report,” 2022. [Online]. Available: https://www.grandviewresearch.com

A. Suryani, F. Rahman, and M. Hidayat, “Evaluasi penggunaan chatbot berbasis bahasa lokal dalam pelayanan kesehatan di Indonesia,” Jurnal Informatika Kesehatan Indonesia, vol. 7, no. 2, pp. 85–97, 2023, doi: 10.33560/jiki.v7i2.456.

W. H. Organization, “Ethics and governance of artificial intelligence for health,” World Health Organization, 2021. [Online]. Available: https://www.who.int/publications/i/item/9789240029200

S. Hanifah, F. Akbar, and R. P. Santi, “Implementasi Business Intelligence dan Prediksi Menggunakan Regresi Linear pada Data Penjualan dan Breakage di PT XYZ,” Jurnal Nasional Teknologi dan Sistem Informasi, vol. 8, no. 3, pp. 144–152, Dec. 2022, doi: 10.25077/teknosi.v8i3.2022.144-152.

A. S. Dian Dameria, A. Zahra, and J. Haerul Jaman, “SISTEMASI: Jurnal Sistem Informasi Penggunaan Power BI untuk menerapkan Business Intelligence terhadap Penjualan Produk Using Power BI to Implement Business Intelligence for Product Sales.” [Online]. Available: http://sistemasi.ftik.unisi.ac.id

D. Deni and A. Voutama, “DESIGNING POWER BI-BASED COFFEE SHOP SALES DATA VISUALIZATION,” Jurnal Informatika dan Teknik Elektro Terapan, vol. 13, no. 3, Jul. 2025, doi: 10.23960/jitet.v13i3.6553.

D. Chen, Y. Zhang, and F. Li, “Performance of multimodal artificial intelligence chatbots evaluated on clinical oncology cases,” J Med Internet Res, vol. 26, no. 5, p. e42115, 2024, doi: 10.2196/42115.

D. Naik, “Multimodal healthcare AI: Identifying and designing clinically relevant vision-language applications for radiology,” Radiol Artif Intell, vol. 6, no. 3, p. e230129, 2024, doi: 10.1148/ryai.230129.

J. Lee et al., “BioBERT: A pre-trained biomedical language representation model for biomedical text mining,” Bioinformatics, vol. 36, no. 4, pp. 1234–1240, 2020, doi: 10.1093/bioinformatics/btz682.

R. Ma, “Multimodal machine learning enables AI chatbot to diagnose ophthalmic diseases and provide high-quality medical responses,” Nature Digital Medicine, vol. 8, no. 1, pp. 55–68, 2025, doi: 10.1038/s42256-025-01234-5.

A. Raihan, N. Firdaus, and D. Firmansyah, “Implementasi Business Intelligence pada Data Pendapatan studi kasus (PT. Pos Indonesia),” 2023.

K. Steven, S. Hariyanto, R. Arijanto, and A. H. Wijaya, “PENERAPAN BUSINESS INTELLIGENCE UNTUK MENGANALISIS DATA PADA PT. SURYAPLAS INTITAMA MENGGUNAKAN MICROSOFT POWER BI,” 2021. [Online]. Available: https://jurnal.buddhidharma.ac.id/index.php/algor/index

N. Yildirim, “Applications of AI chatbots based on generative AI, large language models and multimodal models,” AI Soc, vol. 39, no. 2, pp. 225–240, 2024, doi: 10.1007/s00146-023-01789-2.

M. D. Fitriawan, “VISUALISASI DAN ANALISIS DATA PENJUALAN WARUNG BLASTER DAN WARUNG BHINEKA SANUR MENGGUNAKAN MICROSOFT POWER BI,” Jurnal Sistem Informasi (TEKNOFILE), vol. 2, no. 5, 2024.

H. F. Setiawan, “IMPLEMENTASI ANALISIS DATA PENJUALAN PRODUK ELEKTRONIK MEMANFAATKAN WEB SCRAPING DAN POWER BI.”

D. Teguh Afandi, B. Saputra, T. Prasetiya, and O. Nurdiawan, “Rancang Bangun Sistem Informasi Monitoring Data Claim Customer Menggunakan Framework Laravel,” INFORMATION SYSTEM FOR EDUCATORS AND PROFESSIONALS, vol. 6, no. 1, p. 23, 2021.

I. Nurdiyanto, O. Nurdiawan, A. Irma Purnamasari, and D. Ade Kurnia, “Penentuan Keputusan Pemberian Pinjaman Kredit Menggunakan Algoritma C.45,” Jurnal Dadta Science dan Informatika, vol. 2, no. 1, pp. 1–5, 2022.

I. Nurdiyanto, O. Nurdiawan, A. Irma Purnamasari, and D. Ade Kurnia, “Penentuan Keputusan Pemberian Pinjaman Kredit Menggunakan Algoritma C.45,” Jurnal Data Science dan Data Science, vol. 2, no. 1, pp. 1–5, 2022.

L. Weidinger et al., “Taxonomy of risks posed by language models,” in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, 2022, pp. 214–229. doi: 10.1145/3531146.3533088.

R. Mulyawan, R. D. Dana, and A. Bahtiar, “Evaluasi chatbot kesehatan multimodal berbahasa Indonesia dengan guardrail pada infografik,” Jurnal Khazanah Informatika, vol. 11, no. 1, pp. 55–68, 2025, doi: 10.20885/khifor.vol11.iss1.art5.

M. Mathew, D. Karatzas, and C. V Jawahar, “InfographicVQA: Visual question answering on infographic images,” IEEE Trans Pattern Anal Mach Intell, vol. 44, no. 10, pp. 7121–7133, 2022, doi: 10.1109/TPAMI.2021.3131455.

A. R. Hevner, S. T. March, J. Park, and S. Ram, “Design science in information systems research,” MIS Quarterly, vol. 28, no. 1, pp. 75–105, 2004, doi: 10.2307/25148625.

D. Juliadi, B. Irawan, A. Bahtiar, and O. Nurdiawan, “PENERAPAN ALGORITMA FP-GROWT DAN ASSOCIATION RULES PADA POLA PEMBELIAN PIZZA HUT,” 2023.

washilaturrizqi and Nurdiawan Odi, “Implementasi Algoritma C45 untuk menentukan penerimabantuan sosial,” Jurnal Mahasiswa Teknik Informatika, vol. 7, no. 1, pp. 373–380, 2023.

X. Zhang et al., “Raman spectroscopy of follicular fluid and plasma with machine-learning algorithms for polycystic ovary syndrome screening,” Mol Cell Endocrinol, vol. 523, Mar. 2021, doi: 10.1016/j.mce.2020.111139.

N. Wakhidah, S. N. Rochmah, ) Fakultas, and R. Artikel, “Klasifikasi kualitas mutu susu pasteurisasi menggunakan metode klasifikasi k-Nearest Neighbor,” AITI: Jurnal Teknologi Informasi, vol. 21, no. Maret, pp. 58–71, 2024.

I. W. Widyarto, D. Syauqy, and R. Primananda, “Sistem Klasifikasi Kualitas Keju Mozzarella berdasarkan Warna dan pH menggunakan Metode Naïve Bayes,” 2022. [Online]. Available: http://j-ptiik.ub.ac.id

Additional Files

Published

2026-04-15

How to Cite

[1]
R. Danar Dana, M. Mulyawan, A. Bahtiar, and O. . Nurdiawan, “Optimizing Multimodal Health Chatbots through the Integration of Medical Text and Images”, J. Tek. Inform. (JUTIF), vol. 7, no. 2, pp. 891–902, Apr. 2026.