Usability Evaluation Methods and Their Contribution to UI/UX Design in e-Government: A Systematic Literature Review

Sopian Amir; Harry Budi Santoso; Baginda Anggun Nan Cenka; Nandhita Zefania Maharani; Shabrina Salsabila Kurniawan; Lina Fitria

doi:10.52436/1.jutif.2026.7.4.5587

Authors

Sopian Amir Faculty of Computer Science, Universitas Indonesia, Indonesia
Harry Budi Santoso Faculty of Computer Science, Universitas Indonesia, Indonesia
Baginda Anggun Nan Cenka Faculty of Computer Science, Universitas Indonesia, Indonesia
Nandhita Zefania Maharani Faculty of Computer Science, Universitas Indonesia, Indonesia
Shabrina Salsabila Kurniawan Faculty of Computer Science, Universitas Indonesia, Indonesia
Lina Fitria Faculty of Computer Science, Universitas Indonesia, Indonesia

DOI:

https://doi.org/10.52436/1.jutif.2026.7.4.5587

Keywords:

e-Government, Human-computer Interaction, PRISMA, UI/UX Design, Usability Evaluation

Abstract

Indonesia's digital government transformation has increased efficiency and transparency. However, usability issues continue to hinder the effectiveness of e-government services. These challenges highlight the need for appropriate usability evaluation methods to inform evidence-based UI/UX design improvements. This study conducted a systematic literature review following the PRISMA 2020 guidelines. The review examined usability evaluation methods applied to e-government applications and analysed how the identified usability issues translate into UI/UX design recommendations. Articles were selected based on defined inclusion and exclusion criteria, focusing on empirical studies published between 2020 and 2025. A total of 22 studies were included in the final synthesis. The findings demonstrate a strong reliance on a multi-method usability evaluation approach, specifically a combination of System Usability Scale (SUS), User Experience Questionnaire (UEQ), with usability testing, think-aloud, heuristic evaluation, and eye tracking. Common usability issues include unclear visual hierarchies, complex navigation structures, inadequate feedback mechanisms, and high cognitive load. These issues were systematically synthesized based on usability goals, providing structured insights into recurring design improvement patterns. This study provides an analytical synthesis connecting usability evaluation issues across multiple methods with UI/UX design recommendations, extending previous studies that primarily focused on method identification. The findings offer evidence-based guidance for designers and policymakers to improve the usability of e-government services. However, the study has limitations due to its geographic concentration. Future research is needed to evaluate internal government applications and conduct longitudinal studies assessing the impact of UI/UX improvements on the adoption of public services.

Downloads

Download data is not yet available.

References

D. Mutiarin, N. Khaerah, A. V. I. Nyssa, and N. Nasrulhaq, “E-Government Development: Catalysing Agile Governance Transformation in Indonesia,” Journal of Contemporary Governance and Public Policy, vol. 5, no. 1, pp. 87–110, 2024, doi: 10.46507/jcgpp.v5i1.270.

United Nations, E-Government Survey 2024. 2024.

I. R. Dewi, “27.400 Aplikasi Pemerintah Gabung Jadi 1, Bisa Cari Semuanya.” [Online]. Available: https://www.cnbcindonesia.com/tech/20230824140638-37-465821/27400-aplikasi-pemerintah-gabung-jadi-1-bisa-cari-semuanya?

P. Sukmasetya, H. B. Santoso, and D. I. Sensuse, “Current E-Government Public Service on User Experience Perspective in Indonesia,” 2018 International Conference on Information Technology Systems and Innovation, ICITSI 2018 - Proceedings, pp. 159–164, 2018, doi: 10.1109/ICITSI.2018.8695962.

S. W. Gusman, “Development of the Indonesian Government’s Digital Transformation,” Dinasti International Journal of Education Management And Social Science, vol. 5, no. 5, pp. 1128–1141, 2024, doi: 10.38035/dijemss.v5i5.2868.

Kementerian Sekretariat Negara Republik Indonesia, “Peraturan Presiden Nomor 95 Tahun 2018 tentang Sistem Pemerintahan Berbasis Elektronik,” Menteri Hukum Dan Hak Asasi Manusia Republik Indonesia, p. 110, 2018.

S. A. Ramadhan and U. Pribadi, “Building Citizen Satisfaction with E-Government Services: A Case Study of the Population Administration Information System (SIAK),” Jurnal Manajemen Pelayanan Publik, vol. 8, no. 3, pp. 972–988, 2024, doi: 10.24198/jmpp.v8i3.55866.

N. N. Alazemi and A. J. Al-Shehab, “The rest of this paper, section 2 present the overview of e-government and User experience, section 3 discus the factor affecting user experience in e-government, section 4 present User Experience and Public Value of E-government Services , finally the co,” vol. 13, no. 2, pp. 80–83, 2024, [Online]. Available: https://dl.acm.org/doi/pdf/10.5555/3605526.3605529

S. P. Handasari, R. Wulandari, and Haikal, “Evaluation of the Usability and User Experience of the Jaminan Kesehatan Nasional Mobile Application in Indonesia,” Healthc. Inform. Res., vol. 30, no. 4, p. 324, Oct. 2024, doi: 10.4258/HIR.2024.30.4.324.

F. Munandar and H. B. Santoso, “User Interface Improvement by Evaluating Usability and User Experience: Case Study of Indonesia’s Government Financial Management Information System,” The Indonesian Journal of Computer Science, vol. 14, no. 1, Feb. 2025, doi: 10.33022/IJCS.V14I1.4630.

A. M. Y. Kurnia, H. Nugroho, and R. Leandros, “Sedudo Nganjuk Website Usability Analysis Using User Experience Questionnaire,” in 2024 6th International Conference on Cybernetics and Intelligent System (ICORIS), IEEE, Nov. 2024, pp. 1–6. doi: 10.1109/ICORIS63540.2024.10903771.

H. Tolle, R. I. Rokhmawati, and M. T. Eunike, “User experience design of malang city public service information mobile application using human-centered design method,” ACM International Conference Proceeding Series, pp. 200–205, 2020, doi: 10.1145/3427423.3427467.

K. Nainggolan, K. N. Siahaan, F. M. T. Pardosi, and Sunardi, “User Experience Analysis on the Website of North Sumatra Province Using User Experience Questionnaire (UEQ) and Lean UX Methods,” Proceedings of 2023 International Conference on Information Management and Technology, ICIMTech 2023, pp. 591–596, 2023, doi: 10.1109/ICIMTech59029.2023.10278017.

Claudia and D. P. Lestari, “User-Centered Interaction Design for Indonesia’s E-Government Passport Application: The M-Paspor Indonesia Case Study,” Proceedings of 2024 IEEE International Conference on Data and Software Engineering: Data-Driven Innovation: Transforming Industries and Societies, ICoDSE 2024, pp. 235–240, 2024, doi: 10.1109/ICODSE63307.2024.10829902.

Artomo, D. R. Putri, N. D. Puspaningrum, and Suzanna, “Analysis of User Experience in the Emergency Services Portal on Smartphone Applications,” 2025 5th International Conference on Innovative Research in Applied Science, Engineering and Technology, IRASET 2025, pp. 1–5, 2025, doi: 10.1109/IRASET64571.2025.11008051.

A. Subiyakto, D. N. Ramadhan, N. A. Hidayah, M. Q. Huda, Y. Durachman, and Q. Aini, “Redesigning UI/UX of A Complaint Service Mobile Application Using Goal-Directed Method,” 2024 12th International Conference on Cyber and IT Service Management, CITSM 2024, pp. 1–5, 2024, doi: 10.1109/CITSM64103.2024.10775640.

H. Elmunsyah, W. N. Hidayat, H. Suswanto, K. Asfani, N. H. Muflihah, and Kusumadyahdewi, “UX Validation of Village Administration Information System Using User Experience Questionnaire (UEQ) and Usability Testing,” Proceedings - 4th International Conference on Vocational Education and Electrical Engineering: Strengthening Engagement with Communities through Artificial Intelligence Application in Education, Electrical Engineering and Information Technology, ICVEE 202, pp. 1–6, 2021, doi: 10.1109/ICVEE54186.2021.9649749.

S. Aripiyanto, F. E. Muzayyana Agustin, F. P. Saputrie, Y. Durrachman, D. Khairani, and H. T. Sukmana, “Evaluation of User Experience in the Lecture Activity Menu of AIS Mobile for Student Using Heuristic Evaluation, System Usability Scale and User Experience Questionnaire,” 2024 12th International Conference on Cyber and IT Service Management, CITSM 2024, pp. 1–7, 2024, doi: 10.1109/CITSM64103.2024.10775924.

S. U. Masruroh, M. A. A. Rizqullah, D. Khairani, H. T. Sukmana, S. Aripiyanto, and R. A. Putri, “UI/UX Application of Bus Rapid Transit (BRT) System using User-Centered Design Method and Think Aloud Evaluation,” 2024 3rd International Conference on Creative Communication and Innovative Technology, ICCIT 2024, pp. 1–7, 2024, doi: 10.1109/ICCIT62134.2024.10701192.

M. W. Fanani, N. Q. Lishandary, N. F. Saputri, T. L. Anita, and T. Prasandy, “User Experience Analysis Using Usability Testing Method on Online Queue System at Hospital Service,” 2023 IEEE 9th International Conference on Computing, Engineering and Design, ICCED 2023, pp. 1–6, 2023, doi: 10.1109/ICCED60214.2023.10425688.

G. W. Sasmito and M. Fikri Hidayattullah, “The Implementation of User Centered Design Methods on Public Service Mapping Websites,” IOP Conf. Ser. Mater. Sci. Eng., vol. 1077, no. 1, p. 012022, 2021, doi: 10.1088/1757-899x/1077/1/012022.

A. Sabani, H. Deng, and V. Thai, “Evaluating the development of E-government in Indonesia,” ACM International Conference Proceeding Series, pp. 254–258, Jan. 2019, doi: 10.1145/3305160.3305191.

R. Rahmatullah et al., “A study of user satisfaction and net benefits in indonesia through the DeLone and McLean Model for E-Government success,” Discover Sustainability, vol. 6, no. 1, pp. 1–18, Dec. 2025, doi: 10.1007/S43621-025-01645-4/FIGURES/3.

A. Nugroho, P. I. Santosa, and R. Hartanto, “Usability Evaluation Methods of Mobile Applications: A Systematic Literature Review,” Proceeding - 2022 International Symposium on Information Technology and Digital Innovation: Technology Innovation During Pandemic, ISITDI 2022, pp. 92–95, 2022, doi: 10.1109/ISITDI55734.2022.9944401.

S. U. Masruroh, N. A. Rizqy Vitalaya, H. T. Sukmana, I. Subchi, D. Khairani, and Y. Durachman, “Evaluation of Usability and Accessibility of Mobile Application for People with Disability: Systematic Literature Review,” 2022 International Conference on Science and Technology, ICOSTECH 2022, pp. 1–7, 2022, doi: 10.1109/ICOSTECH54296.2022.9829134.

A. F.M, A. Y.A, A. A.O, A. O.D, and A. O, “A Systematic Review on Usability Evaluation for University Websites,” International Journal of Computer Applications Technology and Research, vol. 11, no. 2, pp. 22–28, 2022, doi: 10.7753/ijcatr1102.1003.

R. Lyzara, B. Purwandari, M. F. Zulfikar, H. B. Santoso, and I. Solichah, “E-Government Usability Evaluation: Insights from A Systematic Literature Review,” ACM International Conference Proceeding Series, pp. 249–253, 2019, doi: 10.1145/3305160.3305178.

M. J. Page et al., “The PRISMA 2020 statement: An updated guideline for reporting systematic reviews,” The BMJ, vol. 372, 2021, doi: 10.1136/bmj.n71.

D. Moher et al., “Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement,” PLoS Med., vol. 6, no. 7, 2009, doi: 10.1371/journal.pmed.1000097.

P. C. Sauer and S. Seuring, How to conduct systematic literature reviews in management research: a guide in 6 steps and 14 decisions, vol. 17, no. 5. Springer Berlin Heidelberg, 2023. doi: 10.1007/s11846-023-00668-3.

V. G. de Menezes, G. V. Pedrosa, M. P. P. da Silva, and R. M. da C. Figueiredo, “Evaluation of Public Services Considering the Expectations of Users—A Systematic Literature Review,” Information (Switzerland), vol. 13, no. 4, pp. 1–14, 2022, doi: 10.3390/info13040162.

“Usability Engineering - Jakob Nielsen - Google Books.” Accessed: Dec. 03, 2025. [Online]. Available: https://books.google.com.vc/books?id=95As2OF67f0C&printsec=frontcover&rview=1#v=onepage&q&f=false

M. J. Page et al., “PRISMA 2020 explanation and elaboration: Updated guidance and exemplars for reporting systematic reviews,” The BMJ, vol. 372, 2021, doi: 10.1136/bmj.n160.

D. Li et al., “An eye-tracking-based approach to evaluate the usability of government portal websites in pilot smart cities,” Engineering, Construction and Architectural Management, vol. 32, no. 4, pp. 2369–2396, Mar. 2025, doi: 10.1108/ECAM-09-2023-0932.

Z. Zhang, D. Chang, J. Zhang, and R. Ding, “Eye Tracking-based Usability Evaluation of E-government App Icon Design,” in 2021 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), IEEE, Dec. 2021, pp. 1651–1655. doi: 10.1109/IEEM50564.2021.9672784.

Z. Liu and G. Juan, “Evaluation Method of User Experience of public service system Using Mobile Eye Tracking,” in Proceedings of the 2020 4th International Conference on Electronic Information Technology and Computer Engineering, New York, NY, USA: ACM, Nov. 2020, pp. 976–980. doi: 10.1145/3443467.3443890.

H. O. Al-Sakran and M. A. Alsudairi, “Usability and Accessibility Assessment of Saudi Arabia Mobile E-Government Websites,” IEEE Access, vol. 9, pp. 48254–48275, 2021, doi: 10.1109/ACCESS.2021.3068917.

A. Aldrees and D. Gračanin, “Gender Disparity in the Usability of E-Government Portals: A Case Study of the Saudi Job Seeking Web Portal,” in Communications in Computer and Information Science, vol. 1529 CCIS, 2022, pp. 276–290. doi: 10.1007/978-3-031-04238-6_21.

H. H. Ho, R. Mohd Rasdi, R. Ibrahim, and M. N. Md Khambari, “Developing and evaluating the effectiveness of mobile phone-based career intervention for career competencies of Malaysian public managers: Protocol for a mixed method study,” Internet Interv., vol. 22, no. August, p. 100349, 2020, doi: 10.1016/j.invent.2020.100349.

R. Alabdulrahman, F. Alqahtani, and M. Binkhonain, “Uncovering Usability Issues in Mobile Government Applications: A User-Centered Approach,” IEEE Access, vol. 13, no. October, pp. 169352–169366, 2025, doi: 10.1109/ACCESS.2025.3614165.

N. Rafifing, O. Mphale, and S. D. Asare, “Exploring User perceptions of an E-Government System in Botswana Using System Usability Scale Model,” Proceedings of the IEEE International Conference on Software Engineering and Service Sciences, ICSESS, vol. 2022-Octob, pp. 195–198, 2022, doi: 10.1109/ICSESS54813.2022.9930194.

M. Milosz and M. Chmielewska, “Usability testing of e-government online services using different methods - A case study,” International Conference on Human System Interaction, HSI, vol. 2020-June, pp. 142–146, 2020, doi: 10.1109/HSI49210.2020.9142628.

F. Nordeström, M. Granbom, S. Iwarsson, and M. Zingmark, “Ageing in the right place—usability of a web-based housing counselling service,” Scand. J. Occup. Ther., vol. 31, no. 1, p., 2023, doi: 10.1080/11038128.2023.2294777.

J. Glad, I. G. Raditya, D. W. Sukmaningsih, and W. Kristian, “Usability Evaluation of the Public Transportation Mobile Application: Laboratory and Field Setting,” 11th International Conference on ICT for Smart Society: Integrating Data and Artificial Intelligence for a Resilient and Sustainable Future Living, ICISS 2024 - Proceeding, pp. 1–7, 2024, doi: 10.1109/ICISS62896.2024.10751583.

Y. Rogers, H. Sharp, and J. Preece, “Interaction Design Beyond Human-Computer Interaction, Sixth Edition (Helen Sharp, Jenny Preece, Yvonne Rogers),” Wiley, vol. 6 edition, no. Wiley, p. 720, 2023.

J. Nielsen, Usability Engineering. Academic Press, 1993.