A Cognitive Level Assessment Based on Bloom’s Taxonomy via NLP and Line-By-Line Methods: Focus on Low-Level Cognitive Competency

Shahidatul Arfah Baharudin; Adidah Lajis

doi:10.53840/myjict10-2-226

Pengarang

Shahidatul Arfah Baharudin Universiti Kuala Lumpur - Malaysian Institute of Information Technology
Adidah Lajis Universiti Kuala Lumpur - Malaysian Institute of Information Technology

DOI:

https://doi.org/10.53840/myjict10-2-226

Kata kunci:

Bloom’s Taxonomy, Cognitive Competency, Natural Language Processing, Educational Assessment, Programming

Abstrak

This paper describes the evaluating cognitive skills is crucial for providing insights into a learner's intellectual competence and readiness to engage with various levels of knowledge. This work presents a new methodology for evaluating cognitive levels, focusing on low-level cognitive competencies, using Natural Language Processing (NLP) and line-by-line analysis. One of the most prominent educational frameworks, Bloom’s Taxonomy, classifies cognitive skills into levels, ranging from basic recall to more advanced thinking. However, research has shown that traditional approaches often fail to measure lower-level cognitive processes like remembering or understanding accurately. This gap can be addressed by using NLP techniques to analyze textual feedback on a line-by-line basis. The proposed approach uses advanced NLP algorithms to identify line-by-line code C language programming assessments that align with the cognitive levels described in Bloom's Taxonomy. Each response line is examined to assess whether it corresponds to foundational cognitive skills like knowledge recall and comprehension. This detailed approach allows for more precise differentiation of various cognitive competencies within lower-order thinking. This methodology has been validated through analysis of a large dataset of student responses across multiple disciplines, with results compared to traditional methods. The findings suggest that the NLP-based framework offers a more accurate and scalable alternative to manual grading, particularly in larger classes where personalized grading is challenging. This research benefits the educational community by offering a scalable approach to cognitive assessment that can be integrated into e-learning platforms. In conclusion, this study demonstrates the potential of combining NLP methods with Bloom’s Taxonomy to assess lower-level cognitive skills in a more effective, scalable, and precise way. The line-by-line analysis provides valuable insights into student understanding, supporting evidence-based educational evaluation.

Muat turun

Muat turun data belum tersedia.

Rujukan

Ala-Mutka, K. M. (2005). A Survey of Automated Assessment Approaches for Programming Assignments. Computer Science Education, 15(2), 83–102. https://doi.org/10.1080/08993400500150747

Allamanis, M., Barr, E. T., Devanbu, P., & Sutton, C. (2019). A survey of machine learning for big code and naturalness. ACM Computing Surveys, 51(4). https://doi.org/10.1145/3212695

Anderson, L. W., Krathwohl Peter W Airasian, D. R., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., Raths, J., & Wittrock, M. C. (2001). Taxonomy for_ Assessing a Revision 0F Bl00M’S Tax0N0My 0F Educati0Nal Objectives. https://www.uky.edu/~rsand1/china2018/texts/Anderson-Krathwohl - A taxonomy for learning teaching and assessing.pdf

Chapagain, J., Tamang, L., Banjade, R., Oli, P., & Rus, V. (2022). Automated Assessment of Student Self-explanation During Source Code Comprehension. Proceedings of the International Florida Artificial Intelligence Research Society Conference, FLAIRS, 35. https://doi.org/10.32473/flairs.v35i.130540

Conejo, R., Barros, B., & Bertoa, M. F. (2019). Automated Assessment of Complex Programming Tasks Using SIETTE. IEEE Transactions on Learning Technologies, 12(4), 470–484. https://doi.org/10.1109/TLT.2018.2876249

Hart, R., Hays, B., McMillin, C., Rezig, E. K., Rodriguez-Rivera, G., & Turkstra, J. A. (2023). Eastwood-Tidy: C Linting for Automated Code Style Assessment in Programming Courses. SIGCSE 2023 - Proceedings of the 54th ACM Technical Symposium on Computer Science Education, 1, 799–805. https://doi.org/10.1145/3545945.3569817

Hwang, W.-Y., Shadiev, R., Wang, C.-Y., & Huang, Z.-H. (2012). A pilot study of cooperative programming learning behavior and its relationship with students’ learning performance. Computers & Education, 58(4), 1267–1281. https://doi.org/10.1016/j.compedu.2011.12.009

K. Dalal, M., & A. Zaveri, M. (2011). Automatic Text Classification: A Technical Review. International Journal of Computer Applications, 28(2), 37–40. https://doi.org/10.5120/3358-4633

Masapanta-Carrión, S., & Velázquez-Iturbide, J. Á. (2018). A systematic review of the use of Bloom’s taxonomy in computer science education. SIGCSE 2018 - Proceedings of the 49th ACM Technical Symposium on Computer Science Education, 2018-Janua, 441–446. https://doi.org/10.1145/3159450.3159491

Mayer, R. E. (2002). A taxonomy for computer-based assessment of problem solving. Computers in Human Behavior, 18(6), 623–632. https://doi.org/10.1016/S0747-5632(02)00020-1

Mekterović, I., Brkić, L., & Horvat, M. (2023). Scaling Automated Programming Assessment Systems. Electronics (Switzerland), 12(4). https://doi.org/10.3390/electronics12040942

Mohammedid, M., & Omar, N. (2020). Question classification based on Bloom’s taxonomy cognitive domain using modified TF-IDF and word2vec. PLoS ONE, 15(3), 1–21. https://doi.org/10.1371/journal.pone.0230442

Nguyen, P. H., Tangworakitthaworn, P., & Gilbert, L. (2020). Individual learning effectiveness based on cognitive taxonomies and constructive Alignment. IEEE Region 10 Annual International Conference, Proceedings/TENCON, 2020-Novem, 1002–1006. https://doi.org/10.1109/TENCON50793.2020.9293733

Panthaplackel, S., Gligoric, M., Mooney, R. J., & Li, J. J. (2020). Associating natural language comment and source code entities. AAAI 2020 - 34th AAAI Conference on Artificial Intelligence, 8592–8599. https://doi.org/10.1609/aaai.v34i05.6382

Smith, D. H., Emeka, C., Fowler, M., West, M., & Zilles, C. (2023). Investigating the Effects of Testing Frequency on Programming Performance and Students’ Behavior. In SIGCSE 2023 - Proceedings of the 54th ACM Technical Symposium on Computer Science Education (Vol. 1, Issue 1). Association for Computing Machinery. https://doi.org/10.1145/3545945.3569821

Somers, R., Cunningham-nelson, S., & Boles, W. (2021). Applying natural language processing to automatically assess student conceptual understanding from textual responses. 37(5), 98–115.

Yulianto, S. V., & Liem, I. (2014). Automatic grader for programming assignment using source code analyzer. Proceedings of 2014 International Conference on Data and Software Engineering, ICODSE 2014, 1–4. https://doi.org/10.1109/ICODSE.2014.7062687

Zhu, Q., Luo, X., Liu, F., Gao, C., & Che, W. (2022). A Survey on Natural Language Processing for Programming. ArXiv.