Benchmarking Brain-Training Apps Using DEGREE and Fuzzy Logic: Lumosity vs Elevate

Heri Sismoro; Mei Parwanto  Kurniawan

doi:10.52436/1.jutif.2026.7.2.5646

Authors

Heri Sismoro Manajemen Informatika, Universitas Amikom Yogyakarta, Indonesia
Mei Parwanto Kurniawan Sistem Informasi, Universitas Amikom Yogyakarta, Indonesia

DOI:

https://doi.org/10.52436/1.jutif.2026.7.2.5646

Keywords:

Brain-training apps, DEGREE, Fuzzy scoring, HCI, Usability evaluation

Abstract

This study provides an actionable benchmark of two popular brain-training apps—Lumosity and Elevate—by applying the 14-factor DEGREE framework as a structured UX evaluation tool and using fuzzy scoring to improve interpretability. We recruited 190 Computer Science undergraduates; each participant evaluated both apps, yielding 380 app evaluations using a counterbalanced two-sheet questionnaire. Fourteen factors covering usability, engagement, and perceived learning were rated on a five-point Likert scale. Reliability was strong for both apps (Cronbach’s α = 0.822 for Lumosity; 0.847 for Elevate). Descriptive results showed mid-to-high perceptions overall, with mean scores of 3.51 (Lumosity) and 3.44 (Elevate). Fuzzy aggregation transformed subjective ratings into a 0–1 index, producing overall scores of 0.503 (Lumosity) and 0.490 (Elevate), indicating a small global advantage for Lumosity. At the factor level, Lumosity was slightly higher on most DEGREE dimensions, whereas Elevate showed relative advantages on Learnability and Confidence, suggesting potential benefits for early onboarding and self-efficacy. Overall, the proposed DEGREE–Fuzzy pipeline yields a transparent, reproducible benchmark that translates multi-factor perceptions into decision-ready recommendations for selecting apps aligned with instructional goals.

Downloads

Download data is not yet available.

References

A. Pumpho, R. Boonsinsukh, and K. Srisim, “Mobile app-based cognitive decision-making and memory games enhance cognitive function in older adults with mild cognitive impairment,” no. October, pp. 1–10, 2025, doi: 10.3389/fpsyg.2025.1633043.

S. M. Cha, “Mobile Application Applied for Cognitive Rehabilitation: A Systematic Review,” Life, vol. 14, no. 7, 2024, doi: 10.3390/life14070891.

H. Kusuma, F. S. Rue, R. Y. Rumagit, and G. D. Pratama, “Usability evaluation of Ruangguru online learning mobile application using heuristic method,” Procedia Comput. Sci., vol. 245, no. C, pp. 176–184, 2024, doi: 10.1016/j.procs.2024.10.241.

R. Lege, “A social presence benchmark framework for extended reality (XR) technologies,” Comput. Educ. X Real., vol. 4, no. December 2023, p. 100062, 2024, doi: 10.1016/j.cexr.2024.100062.

M. Bang, C. W. Jang, H. S. Kim, J. H. Park, and H. E. Cho, “Mobile applications for cognitive training: Content analysis and quality review,” Internet Interv., vol. 33, no. July 2022, p. 100632, 2023, doi: 10.1016/j.invent.2023.100632.

T. Mikolajczyk, F. Moldovan, I. Ciobanu, A. Chalupczak, and A. G. Marin, “Brain Research Using Computer Test,” Procedia Technol., vol. 22, no. October 2015, pp. 1113–1120, 2016, doi: 10.1016/j.protcy.2016.01.157.

A. F. Silva et al., “Cognitive Training with Older Adults Using Smartphone and Web-Based Applications: A Scoping Review,” J. Prev. Alzheimer’s Dis., vol. 11, no. 3, pp. 693–700, 2024, doi: 10.14283/jpad.2024.17.

A. Abid et al., “An evaluation framework and comparative analysis of the widely used learning management systems,” PLoS One, vol. 19, no. 12 December, pp. 1–30, 2024, doi: 10.1371/journal.pone.0311111.

M. P. Kurniawan, M. Suyanto, and E. Utami, “DEGREE : Development and Validation of a User Experience Model for Digital Educational Games Using Cronbach ’ s Alpha and Fuzzy Logic,” vol. 6, no. 4, pp. 1831–1846, 2025, doi: https://doi.org/10.52436/1.jutif.2025.6.4.4942.

A. Asadzadeh, H. Shahrokhi, B. Shalchi, Z. Khamnian, and P. Rezaei-Hachesu, “Serious educational games for children: A comprehensive framework,” Heliyon, vol. 10, no. 6, p. e28108, 2024, doi: 10.1016/j.heliyon.2024.e28108.

V. S. Sanjaya, N. V. Tanoto, N. Tandinata, and D. Ramdhan, “Player evaluation and learning outcomes of ‘NusaVenturas’: A serious game for enhancing player knowledge and interest in Indonesian traditional foods,” Procedia Comput. Sci., vol. 245, no. C, pp. 320–329, 2024, doi: 10.1016/j.procs.2024.10.257.

G. Arastoopour Irgens, C. Bailey, T. Famaye, and A. Behboudi, “User experience testing and co-designing a digital game for broadening participation in computing with and for elementary school children,” Int. J. Child-Computer Interact., vol. 42, no. March, p. 100699, 2024, doi: 10.1016/j.ijcci.2024.100699.

Konul Memmedova and Banu Ertuna, “Development of a Fuzzy Likert Scales to Measure Variables in Social Sciences,” Peer Rev., vol. 09, no. 5, pp. 1–11, 2023, [Online]. Available: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4495102

J. Yang, “Fuzzy comprehensive evaluation system and decision support system for learning management of higher education online courses,” Sci. Rep., vol. 15, no. 1, pp. 1–19, 2025, doi: 10.1038/s41598-025-01782-w.

T. Kyriazos and M. Poga, “Exploring Fuzzy Logic as an Alternative Approach in Psychological Scoring Abstract :,” pp. 1–15, 2024, doi: 10.2174/0118743501337527241125044301.

X. He et al., “Mobile Health App for Adolescent Asthma Self-Management: Development and Usability Study of the Pulmonary Education and Knowledge Mobile Asthma Action Plan,” JMIR Form. Res., vol. 9, pp. 1–12, 2025, doi: 10.2196/64212.

Z. Galavi, S. Norouzi, and R. Khajouei, “Heuristics used for evaluating the usability of mobile health applications: A systematic literature review,” Digit. Heal., vol. 10, 2024, doi: 10.1177/20552076241253539.

Z. R. Li et al., “Usability and Effectiveness of eHealth and mHealth Interventions That Support Self-Management and Health Care Transition in Adolescents and Young Adults With Chronic Disease: Systematic Review,” J. Med. Internet Res., vol. 26, no. 1, 2024, doi: 10.2196/56556.

T. Tene, D. F. Vique López, P. E. Valverde Aguirre, N. I. Cabezas Oviedo, C. Vacacela Gomez, and S. Bellucci, “A systematic review of serious games as tools for STEM education,” Front. Educ., vol. 10, no. February, 2025, doi: 10.3389/feduc.2025.1432982.

K. N. K. Fong et al., “Efficacy of computerized cognitive training using mobile devices to promote cognitive functioning in community-dwelling older adults with or without risk of mild cognitive impairment: A multi-centre longitudinal study,” Digit. Heal., vol. 11, 2025, doi: 10.1177/20552076251362117.

R. F. Adler, K. Baez, P. Morales, J. Sotelo, D. Victorson, and S. Magasi, “Evaluating the Usability of an mHealth App for Empowering Cancer Survivors With Disabilities: Heuristic Evaluation and Usability Testing,” JMIR Hum. Factors, vol. 11, no. 1, pp. 1–14, 2024, doi: 10.2196/51522.

S. Li, C. Wang, and Y. Wang, “Fuzzy evaluation model for physical education teaching methods in colleges and universities using artificial intelligence,” Sci. Rep., vol. 14, no. 1, pp. 1–17, 2024, doi: 10.1038/s41598-024-53177-y.

A. Asemi and A. Asemi, “A judgment-based model for usability evaluating of interactive systems using fuzzy Multi Factors Evaluation (MFE),” Appl. Soft Comput., vol. 117, p. 108411, 2022, doi: 10.1016/j.asoc.2022.108411.

W. Husain et al., “Reliability generalization meta-analysis of the internal consistency of the Big Five Inventory (BFI) by comparing BFI (44 items) and BFI-2 (60 items) versions controlling for age, sex, language factors,” BMC Psychol., vol. 13, no. 1, p. 20, 2025, doi: 10.1186/s40359-024-02271-x.

P. A. Edelsbrunner, B. A. Simonsmeier, and M. Schneider, The Cronbach’s Alpha of Domain-Specific Knowledge Tests Before and After Learning: A Meta-Analysis of Published Studies, vol. 37, no. 1. Springer US, 2025. doi: 10.1007/s10648-024-09982-y.

M. Villar-Guevara, J. H. Livia-Segovia, E. E. García-Salirrosas, and I. Fernández-Mallma, “Student Evaluation of Teachers’ Effectiveness (SETE) scale: translation, cross-cultural adaptation and psychometric properties in a Latin American sample,” Front. Educ., vol. 9, no. July, 2024, doi: 10.3389/feduc.2024.1401718.

H. Takahashi et al., “Educational Utility of Clinical Vignettes Generated in Japanese by ChatGPT-4: Mixed Methods Study,” JMIR Med. Educ., vol. 10, pp. 1–13, 2024, doi: 10.2196/59133.

F. Saigí-Rubió, T. Romeu, E. H. Encuentra, M. Guitert, E. Andrés, and E. Reixach, “Design, Implementation, and Analysis of an Assessment and Accreditation Model to Evaluate a Digital Competence Framework for Health Professionals: Mixed Methods Study,” JMIR Med. Educ., vol. 10, 2024, doi: 10.2196/53462.

J. R. Lindner and N. Lindner, “Interpreting Likert type, summated, unidimensional, and attitudinal scales: I neither agree nor disagree, Likert or not,” Adv. Agric. Dev., vol. 5, no. 2, pp. 152–163, 2024, doi: 10.37433/aad.v5i2.351.

M. Carmona-Halty, K. Alarcón-Castillo, C. Semir-González, G. Sepúlveda-Páez, and W. B. Schaufeli, “Burnout Assessment Tool for Students (BAT-S): evidence of validity in a Chilean sample of undergraduate university students,” Front. Psychol., vol. 15, no. December, pp. 1–8, 2024, doi: 10.3389/fpsyg.2024.1434412.

H. Sigmundsson, H. Hauge, J. S. R. Leversen, and M. Haga, “I CAN intervention to increase grit and growth mindset: exploring the intervention for 15-year-olds Norwegian adolescents,” Front. Educ., vol. 9, no. November, 2024, doi: 10.3389/feduc.2024.1438280.

M. Luo, Y. Gu, F. Zhou, and S. Chen, “Implementation of the Observational Medical Outcomes Partnership Model in Electronic Medical Record Systems: Evaluation Study Using Factor Analysis and Decision-Making Trial and Evaluation Laboratory-Best-Worst Methods,” JMIR Med. Informatics, vol. 12, 2024, doi: 10.2196/58498.

C. R. Peter, “statistics : Open for everyone,” p. 353, 2025.

A. Roquet, P. Martinelli, C. Lampraki, and D. S. Jopp, “Internet Use as a Moderator of the Relationship Between Personal Resources and Stress in Older Adults: Cross-Sectional Study,” JMIR Aging, vol. 7, 2024, doi: 10.2196/52555.

P. J. Muhammad Ali, “Investigating the Impact of Min-Max Data Normalization on the Regression Performance of K-Nearest Neighbor with Different Similarity Measurements,” ARO-The Sci. J. Koya Univ., vol. 10, no. 1, pp. 85–91, 2022, doi: 10.14500/aro.10955.

M. Bergmann, “Fuzzy Membership Functions,” An Introd. to Many-Valued Fuzzy Log., no. xi, pp. 309–314, 2012, doi: 10.1017/cbo9780511801129.018.

J. Yang, “Fuzzy comprehensive evaluation system and decision support system for learning management of higher education online courses,” Sci. Rep., vol. 15, no. 1, pp. 1–18, 2025, doi: 10.1038/s41598-025-01782-w.

M. Mizumoto, “Defuzzification methods,” Handb. Fuzzy Comput., pp. 1–7, 2004, doi: 10.1887/0750304278/b438c24.

G. Milutinović, Computational and Visual Tools for Geospatial Multi-Criteria Decision-Making. Faculty of Engineering and Sustainable Development Department of Computer and Geospatial Sciences SE-801 76 Gävle, Sweden, 2020. doi: 10.1061/41022(336)81.

S. P. Baral, P. K. Parida, and S. K. Sahoo, “Fuzzy TOPSIS Approaches for Multi-criteria Decision-Making Problems in Triangular Fuzzy Numbers,” Springer Nat. Singapor, no. February, pp. 1–28, 2023, doi: 10.1007/978-981-19-6631-6.

A. P. Facchino, D. Marchetti, M. Colasanti, L. Fontanesi, and M. C. Verrocchio, “The use of serious games for psychological education and training: a systematic review,” Front. Educ., vol. 10, no. February, 2025, doi: 10.3389/feduc.2025.1511729.

P. N. Kanimalar and R. Balakumar, “The Art of Fuzzy: Crafting Defuzzification with Centroid of Maxima and Minima,” Int. J. Fuzzy Syst., 2025, doi: 10.1007/s40815-025-01996-0.

H. Schilling, S. Hirschberger, and S. Kauffeld, “A focus group study of students’ expectations of digital onboarding tools in higher education,” Front. Educ., vol. 9, no. November, pp. 1–15, 2024, doi: 10.3389/feduc.2024.1287387.

M. Kuppusamy, J. Singh, P. Raman, and N. Abdolrahimi, “Advanced Decision Analytics with Fuzzy Logic Integrating AI and Computational Thinking for Personnel Selection,” J. Fuzzy Ext. Appl., vol. 5, no. 4, pp. 679–699, 2024, doi: 10.22105/jfea.2024.477833.1621.

H. Hidayat, A. Komariah, B. B. Wiyono, and Y. Huda, “Impact of the Use of Fuzzy Comprehensive Evaluation Applications towards Computational Thinking Skill Students in Engineering Education,” Int. J. Inf. Educ. Technol., vol. 15, no. 1, pp. 90–100, 2025, doi: 10.18178/ijiet.2025.15.1.2221.

A. Alharbi et al., “Selection of data analytic techniques by using fuzzy AHP TOPSIS from a healthcare perspective,” BMC Med. Inform. Decis. Mak., vol. 24, no. 1, 2024, doi: 10.1186/s12911-024-02651-8.