Disclosure of Difficulty Distribution of HOTS-Based Test Questions through Rasch Modeling

Abstract

Many researchers or teachers need more and more complete information about the evaluation of the implementation of learning or the ability measurement tools used to find out how much learning outcomes are shown by their students. In this article, it is shown how to analyze questions based on the level of difficulty and suitability of test item items; especially on HOTS (Higher-Order Thinking Skill) based test questions which were developed based on a cognitive hierarchy adopted from Bloom's taxonomy (C4, C5, & C6). The analysis process was carried out based on Sumintono's (2015) explanation of the combination of standard deviation (SD) values and logit average values (Mean). Then perform the criteria for the outfit mean square (MNSQ) value, the Z-standard outfit value (ZSTD) and the point measure correlation value (Pt. Measure Corr) (Boone et al., 2014). The analysis technique was carried out through Rasch modeling assisted by the Winsteps 3.75 application. As for the grouping of difficulty levels according to Sumintono (2015), namely 1) difficult question category (logit value is greater + 1SD); 2) difficult question category (value 0.0 logit +1 SD); 3) easy question category (value 0.0 logit -1 SD); and 4) the category of questions is very easy (value less than -SD), as well as for the criteria used to measure the suitability of item items using, 1) the value of 0.5 <MNSQ <1.5; 2) value -2.0 <ZSTD <+2.0; and 3) the value of 0.4 <PT-Measure Corr <0.85, (Boone et al., 2014). The results of the analysis show that there are variations in the level of difficulty and suitability of HOTS-based test items. The item analyzed had an acceptable level of suitability and was feasible to maintain because all items met these three criteria. Thus, the collection of HOTS-based test questions is in a good category because it can identify students' various abilities in higher-order thinking with varying levels of difficulty.