Comparative Evaluation of LoRA-Based Parameter-Efficient Fine-Tuning for Indonesian Informal Text Normalization: A Study on Qwen3-14B and Mistral-14B

Authors

  • Fatma Cahyani Department of Information System, Institut Teknologi Sepuluh Nopember, Indonesia
  • Febriliyan Samopa Department of Information System, Institut Teknologi Sepuluh Nopember, Indonesia

DOI:

https://doi.org/10.52436/1.jutif.2026.7.5.5758

Keywords:

Indonesian Text Normalization, Large Language Model, LoRA, Mistral3-14B, Parameter-Efficient Fine-Tuning, Qwen3-14B

Abstract

Informal Indonesian text poses challenges for NLP systems due to abbreviations, slang, and incomplete structures that deviate from standard language forms. This study evaluates the effectiveness of parameter-efficient fine-tuning (PEFT) using Low-Rank Adaptation (LoRA) to adapt large language models for informal-to-formal text normalization in a domain-specific context. Two open-weight multilingual models, Qwen3-14B and Mistral3-14B, were fine-tuned on a dataset of 1,233 formal sentences and 6.165 informal variations derived from academic information at Institut Teknologi Sepuluh Nopember (ITS). Evaluation on a held-out test set of 200 informal queries using lexical (BLEU, ROUGE) and semantic (cosine similarity and sentence-transformer-based similarity) metrics. The results show significant improvements after fine-tuning. Qwen3-14B achieved a BLEU score of 0.660, ROUGE-2 of 0.720, cosine similarity of 0.774, and semantic similarity of 0.907, while Mistral3-14B achieved 0.540, 0.624, 0.699, and 0.884, respectively. Additional experiments indicate that varying LoRA rank has minimal impact on performance, suggesting that LoRA configurations are sufficient for this task. These findings demonstrate that LoRA-based parameter-efficient fine-tuning enables effective and computationally efficient adaptation of large language models for Indonesian informal text normalization. This study provides a scalable solution for domain-specific NLP applications in low-resource settings, particularly for languages with high informal variation.

Downloads

Download data is not yet available.

References

M. Moradi and M. Samwald, “Evaluating the Robustness of Neural Language Models to Input Perturbations,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Nov. 2021, pp. 1558–1570. doi: 10.18653/v1/2021.emnlp-main.117.

J. Khan, K. Ahmad, S. K. Jagatheesaperumal, and K. A. Sohn, “Textual variations in social media text processing applications: challenges, solutions, and trends,” Artificial Intelligence Review, vol. 58, no. 3, Mar. 2025, doi: 10.1007/s10462-024-11071-z.

M. F. Shazwan Fadzely, A. Azizan, N. Alias, N. Khairudin, R. Ismail, and N. Zamin, “Malay Noisy Text Normalization in User-Generated Content: A Rule-Based and Levenshtein Distance Approach,” in Proceedings - International Conference on Knowledge and Systems Engineering (KSE), Institute of Electrical and Electronics Engineers Inc., Nov. 2024, pp. 333–340. doi: 10.1109/KSE63888.2024.11063631.

A. Fikri Aji et al., “One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, May 2022, pp. 7226–7249, doi: 10.18653/v1/2022.acl-long.500.

D. Cavar, L. Vetea Mompelat, and M. Abdo, “The Typology of Ellipsis: A Corpus for Linguistic Analysis and Machine Learning Applications,” in Proceedings of the 6th Workshop on Research in Computational Linguistic Typology and Multilingual NLP, Association for Computational Linguistics, Mar. 2024, pp. 46–54. doi: 10.18653/v1/2024.sigtyp-1.6.

G. Khekare, C. Masudi, Y. K. Chukka, and D. P. Koyyada, “Text Normalization and Summarization Using Advanced Natural Language Processing,” in 2nd International Conference on Integrated Circuits and Communication Systems, ICICACS 2024, Institute of Electrical and Electronics Engineers Inc., Feb. 2024, pp. 1–6. doi: 10.1109/ICICACS60521.2024.10498983.

N. Hidayani, T. Mantoro, and M. A. Ayu, “Deep Learning Model for Sentiment Analysis in the Use of Informal Language and Slang On Social Media,” in 2024 10th International Conference on Computing, Engineering and Design (ICCED), Institute of Electrical and Electronics Engineers Inc., Dec. 2024, pp. 1–5. doi: 10.1109/ICCED64257.2024.10983073.

L. Herlina, E. H. Palandi, and L. S. Nurwahidah, “The Influence of Social Media on Language Evolution: A Linguistic Analysis of Emerging Internet Slang,” International Journal of Social Humanities, vol. 3, no. 10, pp. 3064–7592, Oct. 2024, doi: 10.5281/zenodo.14012418.

A. Ahmad Aliero, B. Sulaimon Adebayo, H. Olanrewaju Aliyu, A. Gogo Tafida, B. Umar Kangiwa, and N. Muhammad Dankolo, “Systematic Review on Text Normalization Techniques and its Approach to Non-Standard Words,” International Journal of Computer Applications, vol. 185, no. 33. pp. 44–55, Sep. 2023, doi: 10.5120/ijca2023923106.

N. Nani, “The Influence of Language on Interaction and Communication Within Social Media Platforms,” Focus Journal Language Review, vol. 2, no. 2, pp. 17–24, Dec. 2024, doi: 10.62795/fjlg.v2i2.295.

J. Khan and S. Lee, “Enhancement of Text Analysis Using Context-Aware Normalization of Social Media Informal Text,” Applied Sciences, vol. 11, no. 17, Sep. 2021, doi: 10.3390/app11178172.

H. Touvron et al., “Llama 2: Open Foundation and Fine-Tuned Chat Models,” Jul. 2023, doi: 10.48550/arXiv.2307.09288.

X. Li and Y. Lan, “Large Language Models are Good Annotators for Type-aware Data Augmentation in Grammatical Error Correction,” in 31st International Conference on Computational Linguistics, Association for Computational Linguistics, Jan. 2025, pp. 199–213.

R. Wang, C. Han, J. Lv, K. Chen, D. Wu, and L. Ma, “Research and Application of Parameter-Efficient Fine-Tuning Methods for Pre-trained Large Models,” in 2025 6th International Conference on Artificial Intelligence and Computer Engineering (ICAICE), Nov. 2025, pp. 552–556. doi: 10.1109/ICAICE68195.2025.11382228.

M. Wong, A. Alshehri, S. Kao, and H. He, “PolyNorm: Few-Shot LLM-Based Text Normalization for Text-to-Speech,” in 2025 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Nov. 2025, pp. 77–85. doi: 10.48550/arXiv.2511.03080.

S. Hu et al., “OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Models,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Jul. 2023, pp. 274–281. doi: 10.18653/v1/2023.acl-demo.26.

M. F. R. Khairul and R. S. Perdana, “Arsitektur Sistem Percakapan Otomatis Berbahasa Indonesia dengan Normalisasi Bahasa Informal Menjadi Baku,” Jurnal Teknologi Informasi dan Ilmu Komputer, vol. 11, no. 5, pp. 1009–1016, Oct. 2024, doi: 10.25126/jtiik.2024117984.

D. M. Dyasputro, “Rule-Based Text Normalization Library for Bahasa Indonesia,” in 2025 10th International Conference on Information Technology and Digital Application (ICITDA), Institute of Electrical and Electronics Engineers Inc., Nov. 2025, pp. 1–8. doi: 10.1109/icitda68167.2025.11332325.

A. Wang, C. Liu, J. Yang, and C. Weng, “Fine-tuning large language models for rare disease concept normalization,” Journal of the American Medical Informatics Association, vol. 31, no. 9, pp. 2076–2083, Sep. 2024, doi: 10.1093/jamia/ocae133.

E. J. Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models,” Oct. 2021, doi: 10.48550/arXiv.2106.09685

L. Wang et al., “Parameter-efficient fine-tuning in large language models: a survey of methodologies,” Artificial Intelligence Review, vol. 58, no. 8, Aug. 2025, doi: 10.1007/s10462-025-11236-4.

H. Yao, “Efficient Fine-tuning of Large Language Models Based on Parameter-efficient LoRA Techniques,” in 2025 10th International Conference on Cyber Security and Information Engineering (ICCSIE), Institute of Electrical and Electronics Engineers Inc., Jul. 2025, pp. 18–22. doi: 10.1109/ICCSIE65832.2025.00012.

J. Hu, X. Liao, J. Gao, Z. Qi, H. Zheng, and C. Wang, “Optimizing Large Language Models with an Enhanced LoRA Fine-Tuning Algorithm for Efficiency and Robustness in NLP Tasks,” in 2024 4th International Conference on Communication Technology and Information Technology (ICCTIT), Institute of Electrical and Electronics Engineers Inc., Dec. 2024, pp. 526–530. doi: 10.1109/ICCTIT64404.2024.10928552.

A. Yang et al., “Qwen3 Technical Report,”, May 2025. doi: 10.48550/arXiv.2505.09388.

A. H. Liu et al., “Ministral 3,”, Jan. 2026. doi: 10.48550/arXiv.2601.08584.

S. Cahyawijaya et al., “NusaCrowd: Open Source Initiative for Indonesian NLP Resources,” Association for Computational Linguistics, pp. 13745–13818, Jul. 2023, doi: 10.18653/v1/2023.findings-acl.868.

A. Wegmann, D. Nguyen, and D. Jurgens, “Tokenization is Sensitive to Language Variation,” in Association for Computational Linguistics 2025, Association for Computational Linguistics, Jul. 2025, pp. 10958–10983. doi: 10.48550/arXiv.2502.15343.

M. Kalra and V. Sharma, “Impact of Low Rank Adaptation with Parameter Efficient Fine-Tuning Techniques on Transformer Models,” in 2025 6th International Conference for Emerging Technology (INCET), Institute of Electrical and Electronics Engineers Inc., May 2025, pp. 1–6. doi: 10.1109/INCET64471.2025.11139894.

Additional Files

Published

2026-10-10

How to Cite

[1]
F. Cahyani and F. Samopa, “Comparative Evaluation of LoRA-Based Parameter-Efficient Fine-Tuning for Indonesian Informal Text Normalization: A Study on Qwen3-14B and Mistral-14B”, J. Tek. Inform. (JUTIF), vol. 7, no. 5, pp. 4780–4792, Oct. 2026.