CUSTOMIZATION AND USABILITY TESTING AUTO ESSAY FILE GRADING LMS BASED METACOGNITIVE ASSESSMENT IN ENGINEERING FACULTY

Ridwan Daud Mahande; Jasruddin; Fitrah Asma  Darmawan; Muhammad Ayat  Hidayat

doi:10.20884/1.jutif.2022.3.2.118

Ridwan Daud Mahande Pendidikan Teknik Informatika dan Komputer, Universitas Negeri Makassar, Indonesia
Jasruddin Fisika, Universitas Negeri Makassar, Indonesia
Fitrah Asma Darmawan Pendidikan Vokasi Mekatronika, Universitas Negeri Makassar, Indonesia
Muhammad Ayat Hidayat Pendidikan Teknik Informatika dan Komputer, Universitas Negeri Makassar, Indonesia

DOI: https://doi.org/10.20884/1.jutif.2022.3.2.118

Keywords: Auto Essay Grading, LMS, Metacognitive

Abstract

Online learning assessment and evaluation has been widely used using the Learning Management system (LMS). However, the form of the questions is still based on multiple choice. Evaluation and form of questions in LMS need to be developed by exploring metacognitive thinking. The LMS plugin supports Essay-based questions and assessments, it's just that this plugin needs to be supported with file submissions and metacognitive assessment rubrics. Therefore, this research aims to customize and test the usability of Auto Essay File Grading (AEG) metacognitive assessment based on LMS. This study uses a quantitative descriptive approach conducted at the Faculty of Engineering, Universitas Negeri Makassar, with lecturers and students as respondents. Data were collected through the USE questionnaire and analyzed using descriptive statistics. The results of the analysis of AEG usability testing from the point of view of lecturers and students who have the highest average are a usefulness. For the lowest average of the lecturers is the ease of use and from the student, the side is the ease of learning. These results also prove that the AEG application/plugin needs improvement in terms of ease and practicality of use so that users learn how to use it faster. Recommendations for ease of use and learning are discussed and explained further, along with the improved display, answer keywords, and minimizing irrelevant menus.

Downloads

Download data is not yet available.

References

J. Peter Ganter, The Trends Mapping Study on Innovation in Technical and Vocational Education and Training (TVET). Bonn: the UNESCO-UNEVOC International Centre for Technical and Vocational Education and Training, 2019.

E. Costello, J. Holland, and C. Kirwan, “The future of online testing and assessment : question quality in MOOCs,” Int. J. Educ. Technol. High. Educ., vol. 15, no. 42, pp. 1–14, 2018.

Y. Xiong and H. K. Suen, “Possibilities , challenges and future directions,” Int. Rev. Educ., vol. 64, pp. 241–263, 2018, doi: 10.1007/s11159-018-9710-5.

L. Earl, S. Katz, and WNCP, Rethinking Classroom Assessment with Purpose in Mind. Canada, 2006. doi: 10.4135/9781446214695.

C. Gautreau, “Motivational Factors Affecting the Integration of a Learning Management System by Faculty,” J. Educ. Online, vol. 8, no. 1, pp. 1–25, 2011.

M. Á. Conde, F. J. García-peñalvo, M. J. Rodriguez-Conde, M. Alier, M. J. Casany, and J. Piguillem, “An evolving Learning Management System for new educational environments using 2 . 0 tools,” no. December, pp. 37–41, 2014, doi: 10.1080/10494820.2012.745433.

I. Han and W. S. Shin, “The use of a mobile learning management system and academic achievement of online students,” Comput. Educ., vol. 16, pp. 1–24, 2016, doi: 10.1016/j.compedu.2016.07.003.

B. S. Dias and J. A. Diniz, “Towards an Enhanced Learning Management System for Blended Learning in Higher Education Incorporating Distinct Learners ’ Profiles,” J. Educ. Technol. Soc., vol. 17, no. 1, pp. 307–319, 2014.

S. Valenti, F. Neri, and A. Cucchiarelli, “An Overview of Current Research on Automated Essay Grading,” vol. 2, 2003.

A. B. Firdausiah, D. Oranova, U. L. Yuhana, and T. Kita, “SISTEM PENILAIAN OTOMATIS JAWABAN ESSAY MENGGUNAKAN ONTOLOGI PADA MOODLE,” Telkomnika, vol. 6, no. 3, pp. 167–172, 2008.

D. S. Mcnamara, S. A. Crossley, R. D. Roscoe, L. K. Allen, and J. Dai, “A hierarchical classification approach to automated essay scoring,” Assess. Writ., vol. 23, pp. 35–59, 2015, doi: 10.1016/j.asw.2014.09.002.

D. Shermis, Mark and J. Burstein, Handbook of Automated Essay Evaluation: Current Aplications and New Directions. New York, London, 2013.

F. Wild, C. Stahl, G. Stermsek, and G. Neumann, “Parameters Driving Effectiveness of Automated Essay Scoring with LSA,” vol. 10, pp. 2–6, 2019.

J. H. Flavell, P. H. Miller, and S. A. Miller, Cognitive Development, 4th ed. New Jersey: Prentice Hall, 1979.

Y. Yusuf, R. Rodding, H. Awang, and I. Mukhtar, “Metacognitive Strategies in Promoting the Development of Generic Competences in High TVE in Malaysia,” Pertanika J. Soc. Sci. Humanit., vol. 25, pp. 247–256, 2017.

D. Moshman, “Metacognitive Theories Revisited,” Educ. Psychol. Rev., 2017, doi: 10.1007/s10648-017-9413-7.

E. R. Lai, “Metacognition : A Literature Review Research Report,” Pearson’s Res. Rep., no. April, 2011.

S. De Klerk, B. P. Veldkamp, and T. J. H. M. Eggen, “A framework for designing and developing multimedia- based performance assessment in vocational education,” Educ. Technol. Res. Dev., vol. 66, no. 1, pp. 147–171, 2018, doi: 10.1007/s11423-017-9559-5.

P. F. Wimmers, Innovation and Change in Professional Education 13 Assessing Competence in Professional Performance across Disciplines and Professions, 1st ed. Switzerland: Springer, 2016. doi: 10.1007/978-3-319-30064-1.

A. M. Lund, “Measuring usability with the use questionnaire12,” Usability Interface, vol. 8, no. 2, pp. 3–6, 2001