DEGREE: Development and Validation of a User Experience Model for Digital Educational Games Using Cronbach’s Alpha and Fuzzy Logic
DOI:
https://doi.org/10.52436/1.jutif.2025.6.4.4942Keywords:
Control, Digital educational games, Evaluation model, Fuzzy logic, User experience, UsabilityAbstract
The rapid growth of digital educational games demands an evaluation model that accurately captures user experience and adopts a human-centred approach. This study introduces DEGREE (Digital Educational Game Review and Evaluation Engine), an enhanced model extending MEEGA+ by incorporating two previously underrepresented dimensions: Control and Feedback. Using a quantitative approach, questionnaires were distributed to high school students who actively use Minecraft and Duolingo, yielding 4800 responses.
Reliability analysis via Cronbach’s Alpha revealed that the Player Experience + Control combination achieved the highest score (α = 0.914), while the inclusion of Feedback reduced reliability (α = 0.864), leading to its exclusion in the final model. The DEGREE model consists of two core domains: Usability (Aesthetics, Learnability, Operability, Accessibility) and Player Experience (Focused Attention, Fun, Challenge, Social Interaction, Confidence, Relevance, Satisfaction, Perceived Learning, User Error Protection, Control). Evaluation scores were calculated using the Fuzzy Weighted Average (FWA) method and Mean of Maximum (MoM) defuzzification. The Control dimension emerged as the most influential (0.2735), followed by Fun (0.2664) and Satisfaction (0.2516), highlighting the significance of user agency in digital learning environments. The DEGREE model offers a statistically robust and user-oriented framework for evaluating educational games, delivering actionable insights for developers and educators to design more effective and engaging digital learning experiences. This study contributes a new validated and generalizable evaluation framework that strengthens the theoretical foundation of user experience assessment in educational game design.
Downloads
References
Y. Li, D. Chen, and X. Deng, “The impact of digital educational games on student’s motivation for learning: The mediating effect of learning engagement and the moderating effect of the digital environment,” PLoS One, vol. 19, no. 1 January, pp. 1–21, 2024, doi: 10.1371/journal.pone.0294350.
Y. Gui, Z. Cai, Y. Yang, L. Kong, X. Fan, and R. H. Tai, “Effectiveness of digital educational game and game design in STEM learning: a meta-analytic review,” Int. J. STEM Educ., vol. 10, no. 1, 2023, doi: 10.1186/s40594-023-00424-9.
Q. U. Ain, M. A. Chatti, W. K. Tsoplefack, R. Alatrash, and S. Joarder, “Designing and Evaluating an Educational Recommender System with Different Levels of User Control,” CEUR Workshop Proc., vol. 3815, pp. 29–45, 2024.
L. Wang, A. de Vetten, W. Admiraal, and R. van der Rijst, “Relationship between perceived learner control and student engagement in various study activities in a blended course in higher education,” Educ. Inf. Technol., vol. 30, no. 2, pp. 2463–2484, 2024, doi: 10.1007/s10639-024-12910-w.
C. Gunduzalp, “The Effects of Digital Game-Based Learning in Technology-Oriented Course: A Case Study in the Biochemistry Department,” J. Educ. Sci. Environ. Heal., pp. 42–59, 2024, doi: 10.55549/jeseh.1419320.
M. P. Kurniawan, M. Suyanto, E. Utami, and Kusrini, “Educational Game Learning Quality Evaluation Tool: A Systematic Literature Review,” Proc. - Int. Conf. Informatics Comput. Sci., pp. 419–425, 2024, doi: 10.1109/ICICoS62600.2024.10636925.
T. Wambsganss, I. Benke, A. Maedche, K. Koedinger, and T. Käser, Evaluating the Impact of Learner Control and Interactivity in Conversational Tutoring Systems for Persuasive Writing, vol. 35, no. 2. Springer New York, 2024. doi: 10.1007/s40593-024-00409-x.
P. A. Edelsbrunner, B. A. Simonsmeier, and M. Schneider, The Cronbach’s Alpha of Domain-Specific Knowledge Tests Before and After Learning: A Meta-Analysis of Published Studies, vol. 37, no. 1. Springer US, 2025. doi: 10.1007/s10648-024-09982-y.
S. Zitzmann and G. A. Orona, “Why We Might Still be Concerned About Low Cronbach’s Alphas in Domain-specific Knowledge Tests,” Educ. Psychol. Rev., vol. 37, no. 2, 2025, doi: 10.1007/s10648-025-10015-5.
S. C. Izah, L. Sylva, and M. Hait, “Cronbach’s Alpha: A Cornerstone in Ensuring Reliability and Validity in Environmental Health Assessment,” ES Energy Environ., vol. 23, pp. 1–14, 2024, doi: 10.30919/esee1057.
K. Chrysafiadi, M. Kamitsios, and M. Virvou, “Fuzzy-based dynamic difficulty adjustment of an educational 3D-game,” Multimed. Tools Appl., vol. 82, no. 18, pp. 27525–27549, 2023, doi: 10.1007/s11042-023-14515-w.
A. Primanita et al., “Adaptive Hint Generation for Educational Games Using Fuzzy Logic,” vol. 18, no. 1, pp. 101–110, 2025.
K. Alhumaid and M. Aassali, “Understanding the role of digital information in enhancing education in UAE: An investigation of the factors that drive continuous adoption,” Int. J. Data Netw. Sci., vol. 7, no. 2, pp. 513–522, 2023, doi: 10.5267/j.ijdns.2023.3.019.
C. Llorente‑Cejudo, “Relationship and variation of dimensions in gamified experiences associated with the predictive model using GAMEX,” J. New Approaches Educ. Res., vol. 13, pp. 1–12, 2024, doi: 10.1007/s44322-023-00002-5.
N. Riedel, R. Schulz, V. Kazezian, and T. Weissgerber, “Replacing bar graphs of continuous data with more informative graphics: are we making progress?,” Clin. Sci., vol. 136, no. 15, pp. 1139–1156, 2022, doi: 10.1042/CS20220287.
L. Xiao, K. T. Hau, and M. D. Wang, “Revisiting the Usage of Alpha in Scale Evaluation: Effects of Scale Length and Sample Size,” Educ. Meas. Issues Pract., vol. 43, no. 2, pp. 74–81, 2024, doi: 10.1111/emip.12604.
J. Talbot, V. Setlur, and A. Anand, “Four experiments on the perception of bar charts,” IEEE Trans. Vis. Comput. Graph., vol. 20, no. 12, pp. 2152–2160, 2014, doi: 10.1109/TVCG.2014.2346320.
C. X. B. Yixuan Li, Emery D. Berger, Minsuk Kahng, From Perception to Decision Assessing the Role of Chart Types Affordances in High-Level Decision Tasks, vol. 1, no. 1. Association for Computing Machinery, 2024. [Online]. Available: http://arxiv.org/abs/2410.04686
S. Sohana, “Heatmap Visualization for Monitoring Health of a Large-scale Cloud System,” 2022.
J. Jylhä, “Visual Heatmaps in User Experience Design,” 2024.
F. Davila, F. Paz, and A. Moquillaza, “Usage and Application of Heatmap Visualizations on Usability User Testing: A Systematic Literature Review,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 14032 LNCS, no. July, pp. 3–17, 2023, doi: 10.1007/978-3-031-35702-2_1.
M. S. Alotaibi, “Game-based learning in early childhood education: a systematic review and meta-analysis,” Front. Psychol., vol. 15, no. April, 2024, doi: 10.3389/fpsyg.2024.1307881.
M. Cai and C. D. Epp, “Predicting Cognitive Load Using Sensor Data in a Literacy Game,” 2024, [Online]. Available: http://arxiv.org/abs/2405.05543
J. Pfau and M. S. El-Nasr, “On Video Game Balancing: Joining Player- and Data-Driven Analytics,” pp. 1–25, 2023, [Online]. Available: http://arxiv.org/abs/2308.07576
T. Kojić, M. Vergari, S. Knuth, M. Warsinke, S. Möller, and J. N. Voigt-Antons, Influence of Gameplay Duration, Hand Tracking, and Controller Based Control Methods on UX in VR, vol. 1, no. 1. Association for Computing Machinery, 2024. doi: 10.1145/3652212.3652222.
D. David and A. Hussein, “A Quality Assessment Methodology for Sign Language Mobile Apps Using Fusion Of Enhanced Weighted Mobile App Rating Scale (MARS) and Content Expert Standardized Criteria,” Appl. Data Sci. Anal., vol. 2023, pp. 66–77, 2023, doi: 10.58496/adsa/2023/005.
R. Nasiboglu and E. Nasibov, “Kybernetika Terms of use : EFFICIENCY ANALYSIS OF THE RULE-BASED DEFUZZIFICATION APPROACH TO FUZZY INFERENCE,” vol. 61, no. 1, pp. 109–132, 2025, doi: 10.14736/kyb-2025-1-0109.
J. Yang, “Fuzzy comprehensive evaluation system and decision support system for learning management of higher education online courses,” Sci. Rep., vol. 15, no. 1, pp. 1–18, 2025, doi: 10.1038/s41598-025-01782-w.
S. Shankar, N. Padmashri, N. Shanmugapriya, S. Ramasamy, and P. S. Sruthi, “IntelliFuzz: An Advanced Fuzzy Logic Framework for Dynamic Evaluation of Student Performance in Open-Ended Learning Tasks,” Int. J. Comput. Exp. Sci. Eng., vol. 11, no. 1, pp. 783–791, 2024, doi: 10.22399/ijcesen.911.
T. Kyriazos and M. Poga, “Exploring Fuzzy Logic as an Alternative Approach in Psychological Scoring Abstract :,” pp. 1–15, 2024, doi: 10.2174/0118743501337527241125044301.
P. D. Paraschos and D. E. Koulouriotis, “Fuzzy Logic-Based Dynamic Difficulty Adjustment for Adaptive Game Environments,” Electron., vol. 14, no. 1, 2025, doi: 10.3390/electronics14010146.
K. Muludi, R. Setianingsih, R. Sholehurrohman, and A. Junaidi, “Exploiting nearest neighbor data and fuzzy membership function to address missing values in classification,” PeerJ Comput. Sci., vol. 10, pp. 1–20, 2024, doi: 10.7717/peerj-cs.1968.
M. R. Mohd Jamil et al., “Crafting a blueprint for enhancing emotional well-being in special education post-pandemic: A fuzzy Delphi approach,” Int. J. Adv. Appl. Sci., vol. 11, no. 11, pp. 99–111, 2024, doi: 10.21833/ijaas.2024.11.011.
T. S. Lee, C. H. Wang, and C. M. Yu, “Fuzzy evaluation model for enhancing E-Learning systems,” Mathematics, vol. 7, no. 10, 2019, doi: 10.3390/math7100918.
P. Stiefenhofer, W. Ding, V. Zhang, and X. Liangxun, “Fuzzy evaluations mappings to model opinions: assessing academic quality in technology enhanced learning environments,” Appl. Math. Sci., vol. 17, no. 4, pp. 189–204, 2023, doi: 10.12988/ams.2023.917369.
J. Slamet and Y. Basthomi, “Assessing Gamification-Based Lms for Efl Students: a Self-Directed Learning Framework,” Stud. Linguist. Cult. FLT, vol. 12, no. 2, pp. 100–122, 2024, doi: 10.46687/CVHT3942.
A. Mohebbi, “Enabling learner independence and self-regulation in language education using AI tools: a systematic review,” Cogent Educ., vol. 12, no. 1, p., 2025, doi: 10.1080/2331186X.2024.2433814.
Y. Zhu, R. Zhou, and Y. Zhang, “Designing preschool children’s educational games for enlightenment through decision analysis methods,” Multimed. Tools Appl., vol. 83, no. 32, pp. 78331–78360, 2024, doi: 10.1007/s11042-024-19803-7.
H. Hidayat, A. Komariah, B. B. Wiyono, and Y. Huda, “Impact of the Use of Fuzzy Comprehensive Evaluation Applications towards Computational Thinking Skill Students in Engineering Education,” Int. J. Inf. Educ. Technol., vol. 15, no. 1, pp. 90–100, 2025, doi: 10.18178/ijiet.2025.15.1.2221.
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Mei Parwanto Kurniawan, M. Suyanto, Ema Utami, Kusrini
This work is licensed under a Creative Commons Attribution 4.0 International License.
