Analysis of Polyglot Obfuscation Techniques against ModSecurity in Preventing Cross-Site Scripting (XSS) and SQL Injection Attacks with Experimental Method
DOI:
https://doi.org/10.52436/1.jutif.2025.6.4.5000Keywords:
Core Rule Set (CRS), Cross-Site Scripting, ModSecurity, Polyglot obfuscation, SQL Injection, Web Application Firewall (WAF)Abstract
Internet use has increased every year, as shown by the percentage of internet users in Indonesia reaching 79.50% in 2024. However, security is something that cannot be ignored, especially with the growing number of Cross-Site Scripting (XSS) and SQL Injection Attacks in web platforms. According to OWASP Top 10 report, these two attacks were listed in 2017 and appeared again in the 2021 version, showing that they are still relevant today. In fact, in June 2024, XSS and SQL Injection vulnerabilities were found in a company, PT. XYZ. One way to mitigate these attacks is by using a Web Application Firewall (WAF) such as ModSecurity, which can protect websites from exploitation. However, previous research found that older versions of ModSecurity had weaknesses that could be bypassed with simple obfuscation techniques. This study aims to analyze the effectiveness of the built-in rules in ModSecurity Core Rule Set (CRS) version 4.7 in handling XSS and SQL Injection payloads with polyglot obfuscation, a method that uses complex character encoding to avoid WAF detection. The research was conducted using an experimental method. This study contributes to improve WAF security by testing against modern obfuscation-based attacks, so that security does not rely solely on the default WAF configuration. The results show that all payloads were detected and blocked by ModSecurity with an HTTP 403 response, proving that the CRS 4.7 built-in rules can effectively protect against XSS and SQL Injection threats.
Downloads
References
rebootonline.com, “Website Statistics Report 2024,” Reboot. Accessed: Nov. 14, 2024. [Online]. Available: https://www.rebootonline.com/website-statistics/
M. Arif, “Press Conference Hasil Survei Penetrasi Internet Indonesia 2024,” 2024. [Online]. Available: https://apjii.or.id/berita/d/apjii-jumlah-pengguna-internet-indonesia-tembus-221-juta-orang
A. D. Riyanto, “Hootsuite (We are Social): Data Digital Indonesia 2024,” Feb. 2024. [Online]. Available: https://andi.link/hootsuite-we-are-social-data-digital-indonesia-2024/
M. Irfan, “Analisis Implementasi Kerentanan Website Laboratiorium Jurusan Teknik Informatika dan Komputer Menggunakan OpenVAS dan Acunetix Vulnerabilty Scanner,” Repos. Politek. Negeri Jakarta, 2024, [Online]. Available: https://repository.pnj.ac.id/id/eprint/21267/
S. Alazmi and D. C. De Leon, “A Systematic Literature Review on the Characteristics and Effectiveness of Web Application Vulnerability Scanners,” IEEE Access, vol. 10, pp. 33200–33219, 2022, doi: 10.1109/ACCESS.2022.3161522.
R. Riska and H. Alamsyah, “Penerapan Sistem Keamanan Web Menggunakan Metode Web Aplication Firewall,” J. Amplif. J. Ilm. Bid. Tek. Elektro Dan Komput., vol. 11, no. 1, pp. 37–42, 2021, doi: 10.33369/jamplifier.v11i1.16683.
M. Annas, R. T. Adek, and Y. Afrillia, “Web Application Firewall (WAF) Design to Detect and Anticipate Hacking in Web-Based Applications,” J. Adv. Comput. Knowl. Algorithms, vol. 1, no. 3, p. 52, 2024, doi: 10.29103/jacka.v1i3.16315.
Andreya E., “Antisipasi Bersama Tingkatkan Sistem dan Cegah Serangan Siber,” Sep. 2022. [Online]. Available: https://aptika.kominfo.go.id/2022/09/antisipasi-bersama-tingkatkan-sistem-dan-cegah-serangan-siber/
I. R. Team, “WEB DEFACEMENT : JUDI ONLINE,” JAKARTA, Jun. 2023. Accessed: Nov. 14, 2024. [Online]. Available: https://www.bssn.go.id/langkah-langkah-penanggulangan-insiden-web-defacement-judi-online/
D. Lee, B. Steed, Y. Liu, and O. Ezenwoye, “Tutorial: A Lightweight Web Application for Software Vulnerability Demonstration,” Proc. - 2021 IEEE Secur. Dev. Conf. SecDev 2021, pp. 5–6, 2021, doi: 10.1109/SecDev51306.2021.00014.
A. A. Chandra, A. Turmudi Zy, and A. Nugroho, “PENERAPAN TEKNIK PENETRATION TESTING TERHADAP CROSS SITE SCRIPTING (XSS) DALAM PENGEMBANGAN WEBSITE,” Rabit J. Teknol. dan Sist. Inf. Univrab, vol. 9, no. 2, pp. 262–270, Jul. 2024, doi: 10.36341/rabit.v9i2.4822.
S. Suroto and A. Asman, “Ancaman Terhadap Keamanan Informasi Oleh Serangan Cross-Site Scripting (Xss) Dan Metode Pencegahannya,” Zo. Komput., vol. 11, no. 1, pp. 11–19, 2021, doi: https://doi.org/10.37776/zk.v11i1.658.
A. S. Hakim, T. A. Cahyanto, and H. Azizah, “Serangan cross-site scripting (XSS) berdasarkan base metric CVSS V.2,” J. Smart Teknol., vol. 2, no. 1, 2020, [Online]. Available: http://jurnal.unmuhjember.ac.id/index.php/JST/article/view/3839
M. Liu, B. Zhang, W. Chen, and X. Zhang, “A Survey of Exploitation and Detection Methods of XSS Vulnerabilities,” IEEE Access, vol. 7, pp. 182004–182016, 2019, doi: 10.1109/ACCESS.2019.2960449.
nvd.nist.gov, “CVE-2024-9294 Detail,” National Vulnerability Database. Accessed: Nov. 14, 2024. [Online]. Available: https://nvd.nist.gov/vuln/detail/CVE-2024-9294
Hardianto and T. Subari, “Analisis Cyber Crime handling pada Aplikasi Web dengan WAF ModSecurity,” Petir, vol. 16, no. 1, pp. 91–99, Apr. 2023, doi: 10.33322/petir.v16i1.1910.
G. E. Cárdenas Rosero, C. P. Guevara Vega, and P. Landeta-López, “Website Protection: An Evaluation of the Web Application Firewall,” Data Metadata, vol. 4, 2025, doi: 10.56294/DM2025190.
R. A. Muzaki, O. C. Briliyant, M. A. Hasditama, and H. Ritchi, “Improving Security of Web-Based Application Using ModSecurity and Reverse Proxy in Web Application Firewall,” 2020 Int. Work. Big Data Inf. Secur. IWBIS 2020, pp. 85–90, 2020, doi: 10.1109/IWBIS50925.2020.9255601.
S. D. Utama, F. Ferdian, A. Bahtiar, E. Pratama, and N. Arista, “Web Application Firewall Menggunakan ModSecurity,” IET Inf. Secur., vol. 17, no. 1, pp. 900–926, 2019, [Online]. Available: http://www.sicherheitsforschung-magdeburg.de/uploads/journal/MJS_061_Bijjou_Bypassing.pdf
N. Tewari and G. Datt, “A Study on the Systematic Review of Security Vulnerabilities of Popular Web Browsers,” Proc. Int. Conf. Technol. Adv. Innov. ICTAI 2021, pp. 314–318, 2021, doi: 10.1109/ICTAI53825.2021.9673463.
G. Areo, “Advanced Cybersecurity Strategies for Detecting and Preventing Cross-Site Scripting (XSS) Attacks,” no. November, p. 8, 2024, [Online]. Available: https://www.researchgate.net/publication/385492153
R. A. AlSufaian, K. H. AlQahtani, R. M. AlAjmi, R. A. AlMoussa, R. A. AlGhamdi, and N. A. Saqib, “Web Application Security Using Obfuscation,” no. June, pp. 0–6, 2012.
D. Garg, “Uncovering XSS Polyglot Payload Detection with Machine Learning : Advancing Web Security Against Complex Threats,” pp. 0–19, 2024, doi: https://doi.org/10.21203/rs.3.rs-5564100/v1.
L. Koch et al., “On the Abuse and Detection of Polyglot Files,” WWW 2025 - Proc. ACM Web Conf., pp. 4810–4822, 2025, doi: 10.1145/3696410.3714814.
M. Alagoz, M. S. Tok, and K. Bicakci, “Exploring and Improving the Usability of ModSecurity Web Application Firewall,” 14th Int. Conf. Inf. Secur. Cryptology, ISCTURKEY 2021 - Proc., no. December, pp. 51–56, 2021, doi: 10.1109/ISCTURKEY53027.2021.9654294.
E. Mulyo, “Analisa WAF (Web Application Firewall) Menggunakan Nginx Terhadap Serangan Sql Injection,” 2021, [Online]. Available: http://digilib.mercubuana.ac.id/
OWASP, “XSS Filter Evasion Cheat Sheet.” [Online]. Available: https://cheatsheetseries.owasp.org/cheatsheets/XSS_Filter_Evasion_Cheat_Sheet.html
E. Malays and S. Sakti, “Analisis pengamanan Website dari Serangan Cross Site Script ( XSS ) dengan htmlspecialchars dan strip _ tags,” vol. 25, no. 1, pp. 177–183, 2024, doi: https://doi.org/10.37817/tekinfo.v25i1.
D. G. Sembiring, “Apa itu Serangan Cross-Site Scripting (XSS)?,” 2024, Information Technology Certification Center, Jakarta Barat. [Online]. Available: https://itcc.itpln.ac.id/apa-itu-serangan-cross-site-scripting-xss/
M. D. Khoiroh et al., “PENETRATION TESTING UNTUK MENGUJI KERENTANAN SISTEM INFORMASI PEMERINTAH DAERAH,” J. Sist. dan Teknol. Inf., vol. 06, no. 2, pp. 1–5, 2024.
S. Tahiri, “How we bypassed libModSecurity aka ModSecurity,” Linkedin Article. Accessed: Nov. 16, 2024. [Online]. Available: https://www.linkedin.com/pulse/how-we-bypassed-libmodsecurity-aka-modsecurity-soufiane-tahiri/
W. El Labban, “Bypassing a Web Application Firewall,” 2024. Accessed: Nov. 15, 2024. [Online]. Available: https://wissam-labban.com/Projects/Modesecurity WAF Bypassing.pdf
qazbnm456, “Bypass the latest CRS v3.1.0 rules of SQL injection,” github.com.
A. Chowdhary, K. Jha, and M. Zhao, “Generative Adversarial Network (GAN)-Based Autonomous Penetration Testing for Web Applications,” Sensors, vol. 23, no. 18, Sep. 2023, doi: 10.3390/s23188014.
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Nelmiawati, Kessy Dealova
This work is licensed under a Creative Commons Attribution 4.0 International License.
