From Logs to Insights in the Pulp & Paper Industry: Generating Structured Alarm Reports Using LLMs and RAG
DOI:
https://doi.org/10.52436/1.jutif.2025.6.5.5225Keywords:
Alarm Management, Large Language Model, Retrieval-Augmented Generation, Semantic Search, Text EmbeddingAbstract
Effective alarm management is essential in industrial environments to ensure operational safety and minimize costly downtime. Traditional rule-based reporting systems often struggle to handle heterogeneous alarm log formats and the complexity of natural language queries, limiting their adaptability in real-world applications. To address these limitations, this study proposes a generative alarm reporting system that integrates Large Language Models (LLMs) with a Retrieval-Augmented Generation (RAG) framework. The system converts natural language queries into structured JSON filters, enabling efficient retrieval of contextual information from historical alarm logs. Three open-source LLMs—CodeLlama-7B, LLaMA 3.1-8B, and Mistral-7B—were locally deployed and evaluated using both quantitative and qualitative methods. Experimental results show that CodeLlama-7B achieved the best overall performance, with an Exact Match Accuracy of 0.80, a Field Match score of 93.8%, and a 0% Parse Failure Rate, outperforming the other models in reliability and structural consistency. Compared to conventional rule-based approaches, the proposed LLM-RAG integration demonstrates improved relevance, interpretability, and responsiveness in alarm reporting. This work represents the first systematic benchmarking of locally deployed open-source LLMs for industrial alarm management, providing a replicable framework and highlighting their potential to advance intelligent, real-time, and domain-specific reporting in the pulp and paper industry and beyond.
Downloads
References
ISA, ISA-18.2 Standard: Management of Alarm Systems, 2016.
IEC, IEC 62682: Management of Alarm Systems for the Process Industries, 2014.
B. Hollifield and E. Habibi, The Alarm Management Handbook, 2nd ed., ISA, 2015. [Online]. Available: https://www.isa.org/getmedia/b360ad22-4c7b-4859-bfd4-8de6b076cc9e/Alarm-Management-2nd-Ed_Hollifield-Habibi_CH1-Final-3-15.pdf.
S. Akhtar, et al., “Economic impact of alarm flooding,” Journal of Industrial Safety Engineering, 2025.
Parsa, K., Hassall, M., & Naderpour, M. (2021). Process Alarm Modeling Using Graph Theory: Alarm Design Review and Rationalization. In IEEE Systems Journal (Vol. 15, Issue 2, pp. 2257–2268). Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/JSYST.2020.3019041
Manca, G., & Fay, A. (2021). Detection of Historical Alarm Subsequences Using Alarm Events and a Coactivation Constraint. IEEE Access, 9, 46851–46873. https://doi.org/10.1109/ACCESS.2021.3067837
Andrade, J. R., Rocha, C., Silva, R., Viana, J. P., Bessa, R. J., Gouveia, C., Almeida, B., Santos, R. J., Louro, M., Santos, P. M., & Ribeiro, A. F. (2022). Data-Driven Anomaly Detection and Event Log Profiling of SCADA Alarms. IEEE Access, 10, 73758–73773. https://doi.org/10.1109/ACCESS.2022.3190398
A. Raveendran, B. Radhakrishnan, and R. Sivakumar, “Best Practices in Alarm Rationalization: A Data-Driven Approach,” ISA Transactions, vol. 102, pp. 390–402, Jul. 2020, doi: 10.1016/j.isatra.2020.02.006.
Hindy, H., Brosset, D., Bayne, E., Seeam, A., & Bellekens, X. (2019). Improving SIEM for Critical SCADA Water Infrastructures Using Machine Learning. https://doi.org/10.1007/978-3-030-12786-2_1.
Schummer, P., del Rio, A., Serrano, J., Jimenez, D., Sánchez, G., & Llorente, Á. (2024). Machine Learning-Based Network Anomaly Detection: Design, Implementation, and Evaluation. AI, 5(4), 2967-2983. https://doi.org/10.3390/ai5040143
Z. Jiang, L. Wang, and M. Zhou, “Evaluation of Industrial NLP for Process Safety: Applications to Alarm Logs,” Computers & Chemical Engineering, vol. 172, p. 108184, Apr. 2023, doi: 10.1016/j.compchemeng.2023.108184.
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2023). Retrieval-Augmented Generation for Large Language Models: A Survey. http://arxiv.org/abs/2312.10997
W. Zhao, X. Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., Du, Y., Yang, C., Chen, Y., Chen, Z., Jiang, J., Ren, R., Li, Y., Tang, X., Liu, Z., Wen, J.-R. (2023). A Survey of Large Language Models. ArXiv. http://arxiv.org/abs/2303.18223
T. Brown, et al., “Language Models are Few-Shot Learners,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 33, pp. 1877–1901, 2020.
OpenAI, “GPT-4 Technical Report,” arXiv preprint arXiv:2303.08774, 2023.
S. Bubeck, et al., “Sparks of Artificial General Intelligence: Early experiments with GPT-4,” arXiv preprint arXiv:2303.12712, 2023.
P. Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” in Advances in Neural Information Processing Systems (NeurIPS), 2020
B. Ni, Z. Liu, L. Wang, Y. Lei, Y. Zhao, X. Cheng, Q. Zeng, L. Dong, Y. Xia, K. Kenthapadi, R. Rossi, F. Dernoncourt, M. M. Tanjim, N. Ahmed, X. Liu, W. Fan, E. Blasch, Y. Wang, M. Jiang, and T. Derr, “Towards Trustworthy Retrieval-Augmented Generation for Large Language Models: A Survey,” arXiv preprint arXiv:2404.10968, Apr. 2024.
V. Karpukhin et al., “Dense Passage Retrieval for Open-Domain Question Answering,” in Proc. EMNLP, 2020.
Q. Zhang et al., “VBase: Unifying Online Vector Similarity Search and Relational Queries via Relaxed Monotonicity,” in Proc. 17th USENIX OSDI, 2023.
Michel, D. D. E., Clovis, T. N., Christian, T. T., Mohamadou, A., & Sone, M. E. (2023). Machine Learning-Based Alarms Classification and Correlation in an SDH/WDM Optical Network to Improve Network Maintenance. Journal of Computer and Communications, 11(02), 122–141. https://doi.org/10.4236/jcc.2023.112009
Rodrigo, V., Chioua, M., Hagglund, T., & Hollender, M. (2016). Causal analysis for alarm flood reduction.
S. Dey, “Effective Semantic Search: Vector Databases in the LLM Era,” Medium, Dec. 7, 2024. [Online]. Available: https://medium.com/@soumavadey/effective-semantic-search-vector-databases-in-the-llm-era-5720f1bf0bbf
P. Gupta et al., “Evaluating Structured Output in Generative Models: A Case for Field-Level Matching,” in Proc. AAAI Conf. Artificial Intelligence, vol. 37, no. 13, pp. 14972–14980, 2023.
M. Biesialska, K. Biesialska, and J. M. Pino, “Evaluation of Machine Translation Output for Syntactic Correctness,” Computational Linguistics, vol. 47, no. 4, pp. 789–822, 2021.
L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica, “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena,” *arXiv preprint*, arXiv:2306.05685v4, Dec. 2023.
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Handri Santoso, Oktavianus Hendry Wijaya, Febri Andriani, Sonny Prijantono
This work is licensed under a Creative Commons Attribution 4.0 International License.
