EVALUATION OF USER EXPERIENCE (UX) HNI MOBILE USING THE THINK ALOUD METHOD

Muhammad Dio Revansa; Angraini Angraini; Tengku Khairil Ahsyar; Syaifullah Syaifullah

doi:10.52436/1.jutif.2024.5.4.2120

Muhammad Dio Revansa Information System, Faculty of Science and Technology, Universitas Islam Negeri Sultan Syarif Kasim Riau, Indonesia
Angraini Information System, Faculty of Science and Technology, Universitas Islam Negeri Sultan Syarif Kasim Riau, Indonesia
Tengku Khairil Ahsyar Information System, Faculty of Science and Technology, Universitas Islam Negeri Sultan Syarif Kasim Riau, Indonesia
Syaifullah Information System, Faculty of Science and Technology, Universitas Islam Negeri Sultan Syarif Kasim Riau, Indonesia

DOI: https://doi.org/10.52436/1.jutif.2024.5.4.2120

Keywords: Evaluation, Mobile Application, Think Aloud, User Experience

Abstract

User experience (UX) evaluation is important to determine the quality of a product or service. This research evaluates the UX of the HNI Mobile application using the think aloud method which asks users to express their thoughts while using the system. Involving 5 respondents who performed 10 task scenarios. The respondents' verbalization data, transcribed, and analyzed with NVIVO 12. The results obtained 5 main codes/themes of problems, namely profile editing, language features, member notifications, feature layout, and feature upgrades, which were visualized in a mind map. A retrospective probing technique was used to understand user difficulties. Although the main features functioned well, some respondents experienced difficulties in using the supporting features such as edit profile, change language, and access notifications. These findings provide input to improve the user experience of the HNI Mobile app in the future.

Downloads

Download data is not yet available.

References

I. Díaz-Oreiro, G. López, L. Quesada, and L. Guerrero, “Standardized Questionnaires for User Experience Evaluation: A Systematic Literature Review,” no. October 2018, p. 14, 2019, doi: 10.3390/proceedings2019031014.

“user experience - Pelajari - Google Trends.” https://trends.google.co.id/trends/explore?date=today 5-y&q=user experience&hl=id (accessed Mar. 30, 2024).

Mochammad Aldi Kushendriawan, Harry Budi Santoso, Panca O. Hadi Putra, and Martin Schrepp, “Evaluating User Experience of a Mobile Health Application ‘Halodoc’ using User Experience Questionnaire and Usability Testing,” J. Sist. Inf., vol. 17, no. 1, pp. 58–71, 2021, doi: 10.21609/jsi.v17i1.1063.

B. Alsanousi, A. S. Albesher, H. Do, and S. Ludi, “Investigating the User Experience and Evaluating Usability Issues in AI-Enabled Learning Mobile Apps: An Analysis of User Reviews,” Int. J. Adv. Comput. Sci. Appl., vol. 14, no. 6, pp. 18–29, 2023, doi: 10.14569/IJACSA.2023.0140602.

A. G. Imana and Y. S. Nugroho, “Ux (User Experience) Evaluation of the Openlearning System At Universitas Muhammadiyah Surakarta Using Heuristic Evaluation and Usability Testing,” J. Tek. Inform., vol. 4, no. 4, pp. 681–691, 2023, doi: 10.52436/1.jutif.2023.4.4.824.

C. Cheng and C. Tristan, “Integrating heuristics and think ‑ aloud approach to evaluate the usability of game ‑ based learning material,” vol. 8, pp. 137–157, 2021.

Z. A. Nasruddin, A. Markom, and M. Abdul Aziz, “Evaluating construction defect mobile app using think aloud,” Commun. Comput. Inf. Sci., vol. 886, no. 2, pp. 3–11, 2018, doi: 10.1007/978-981-13-1628-9_1.

A. K. Trukenbrod, N. Backhaus, and R. Thomaschke, “Measuring subjectively experienced time in usability and user experience testing scenarios,” Int. J. Hum. Comput. Stud., vol. 138, no. January, p. 102399, 2020, doi: 10.1016/j.ijhcs.2020.102399.

J. G. Smith, N. S. Alamiri, G. Biegger, C. Frederick, J. P. Halbert, and K. S. Ingersoll, “Think-Aloud Testing of a Novel Safer Drinking App for College Students During COVID-19: Usability Study,” JMIR Form. Res., vol. 6, no. 2, 2022, doi: 10.2196/32716.

D. Szinay, O. Perski, A. Jones, T. Chadborn, J. Brown, and F. Naughton, “Influences on the uptake of health and well-being apps and curated app portals: Think-aloud and interview study,” JMIR mHealth uHealth, vol. 9, no. 4, 2021, doi: 10.2196/27173.

D. Nakikj, D. Kreda, and N. Gehlenborg, “New Ways for Patients to Make Sense of Their Electronic Health Record Data Using the Discovery Web Application: Think-Aloud Evaluation Study,” JMIR Form. Res., vol. 7, 2023, doi: 10.2196/41346.

K. Ishaq, F. Rosdi, N. A. M. Zin, and A. Abid, “Heuristic And Think Aloud Method To Evaluate The Low Fidelity Prototype Of Game-Based Language Learning Application,” 4th Int. Conf. Innov. Comput. ICIC 2021, no. Icic, 2021, doi: 10.1109/ICIC53490.2021.9693022.

Z. Tupikovskaja-Omovie, “Enhancing User Experience in Fashion m-Retail: Mapping Shopping User Journey Using Google Analytics, Eye Tracking Technology and Retrospective Think Aloud Interview,” Fash. Pract., vol. 14, no. 3, pp. 352–375, 2022, doi: 10.1080/17569370.2022.2129466.

H. Virtanen, “Methods for evaluating user experience in small agile teams: a case study,” 2022.

E. Oliveira et al., “End-user Evaluation of a Mobile Application Prototype for Territorial Innovation,” Int. Conf. Enterp. Inf. Syst. ICEIS - Proc., vol. 2, no. Iceis, pp. 495–504, 2021, doi: 10.5220/0010479104950504.

M. Ahmed, K. M. Ying, and K. E. Boyer, “User-centered design of a mobile java practice app: A comparison of question formats,” SIGCSE 2020 - Proc. 51st ACM Tech. Symp. Comput. Sci. Educ., pp. 1158–1164, 2020, doi: 10.1145/3328778.3366942.

R. N. Lindstrom and A. De Havilland, Conducting the review of demonstrations for higher operating value for landfill wells. 2017.

K. M. Staller, “Big enough? Sampling in qualitative inquiry,” Qual. Soc. Work, vol. 20, no. 4, pp. 897–904, 2021, doi: 10.1177/14733250211024516.

B. Noushad, P. W. M. Van Gerven, and A. B. H. de Bruin, “Twelve tips for applying the think-aloud method to capture cognitive processes,” Med. Teach., vol. 0, no. 0, pp. 1–6, 2023, doi: 10.1080/0142159X.2023.2289847.

P. M. Zaini et al., Metodologi Penelitian Kualitatif, no. May. 2023.

J. H. Birns, K. a Joffre, J. F. Leclerc, and C. A. Paulsen, “Getting the Whole Picture : Collecting Usability Data Using Two Methods –– Concurrent Think Aloud and Retrospective Probing,” Usability Prof. Assoc. Conf. July, pp. 8–12, 2002.

S. Ntoa, G. Margetis, M. Antona, and C. Stephanidis, “User Experience Evaluation in Intelligent Environments: A Comprehensive Framework,” Technologies, vol. 9, no. 2, 2021, doi: 10.3390/technologies9020041.

C. Neilson and P. Grigore, “Machine Learning and AI Application Behaviour Prediction for User Experience Modelling and Optimization,” J. Comput. Nat. Sci., vol. 2, no. 3, pp. 120–131, 2022, doi: 10.53759/181x/jcns202202015.

G. S. C. A. B. N. H. M. Z. , Muhammad Omar, Farhana Diana Deris, “User Experience in Product Design and Development: Perspectives and Strategies,” Math. Stat. Eng. Appl., vol. 71, no. 2, pp. 257–262, 2022, doi: 10.17762/msea.v71i2.83.

M. D. Wolcott and N. G. Lobczowski, “Using cognitive interviews and think-aloud protocols to understand thought processes,” Curr. Pharm. Teach. Learn., no. xxxx, pp. 0–1, 2020, doi: 10.1016/j.cptl.2020.09.005.

T. Vanicek and S. Popelka, “The Think-Aloud Method for Evaluating the Usability of a Regional Atlas,” ISPRS Int. J. Geo-Information, vol. 12, no. 3, 2023, doi: 10.3390/ijgi12030095.

I. M. Candiasa, I. G. A. Gunadi, and I. N. W. S. Putra, “UX Evaluation Using Firstclick, Performance Measurement, RTA, And Questionnaire On E-Commerce Website,” Sinkron, vol. 8, no. 1, pp. 451–460, 2023, doi: 10.33395/sinkron.v8i1.12037.