Analyzing Students’ Conceptual Understanding in Scientific Writing Using One-Parameter Logistic Item Response Theory

Abstract

This study evaluated students' conceptual understanding of academic writing using the Parameter Logistic Model (1PL) of Item Response Theory (IRT). It involved 135 students selected through stratified random sampling. A test consisting of 25 multiple-choice questions was designed to assess key aspects of academic writing, problem formulation, reference management, use of academic language, and structural coherence. Data were analyzed using WINSTEPS software to determine the difficulty level and students' ability to understand the items.

The results showed that the average total score was 12.9, with a maximum score of 24 and a minimum score of 1. The average ability measure was 0.11 logit, slightly above the average logit of the instrument, indicating moderate conceptual understanding. The standard deviation of the ability measure was 1.98, reflecting substantial variation in student skills. Item difficulty levels ranged from -1.88 logit (easiest) to 2.23 logit (most difficult), with an average difficulty level of 0.00 logit. The instrument's reliability was high, with a person reliability index of 0.90 and a separation index of 2.96, indicating that the test effectively differentiated students with low, medium, and high ability levels.

This considers students' scholastic composing abilities, especially in quotation exactness and consistent structure. It underlines the significance of creating focused-on-educating techniques to address particular shortcomings in students' scholarly composing aptitudes. The findings contribute to improving curriculum design and pedagogical approaches in higher education, ensuring more evidence-based and contextually relevant teaching methodologies.