Disclosure of Difficulty Distribution of HOTS-Based Test Questions through Rasch Modeling

Ade Yulianto, Ari Widodo


Many researchers or teachers need more and more complete information about the evaluation of the implementation of learning or the ability measurement tools used to find out how much learning outcomes are shown by their students. In this article, it is shown how to analyze questions based on the level of difficulty and suitability of test item items; especially on HOTS (Higher-Order Thinking Skill) based test questions which were developed based on a cognitive hierarchy adopted from Bloom's taxonomy (C4, C5, & C6). The analysis process was carried out based on Sumintono's (2015) explanation of the combination of standard deviation (SD) values and logit average values (Mean). Then perform the criteria for the outfit mean square (MNSQ) value, the Z-standard outfit value (ZSTD) and the point measure correlation value (Pt. Measure Corr) (Boone et al., 2014). The analysis technique was carried out through Rasch modeling assisted by the Winsteps 3.75 application. As for the grouping of difficulty levels according to Sumintono (2015), namely 1) difficult question category (logit value is greater + 1SD); 2) difficult question category (value 0.0 logit +1 SD); 3) easy question category (value 0.0 logit -1 SD); and 4) the category of questions is very easy (value less than -SD), as well as for the criteria used to measure the suitability of item items using, 1) the value of 0.5 <MNSQ <1.5; 2) value -2.0 <ZSTD <+2.0; and 3) the value of 0.4 <PT-Measure Corr <0.85, (Boone et al., 2014). The results of the analysis show that there are variations in the level of difficulty and suitability of HOTS-based test items. The item analyzed had an acceptable level of suitability and was feasible to maintain because all items met these three criteria. Thus, the collection of HOTS-based test questions is in a good category because it can identify students' various abilities in higher-order thinking with varying levels of difficulty.


Analysis of Item Difficulty and Suitability; HOTS-Based Test Questions; Rasch Modeling.

Full Text:



Alhamdulillah, et al. (1996). Glossary of educational Assessment Terms. Jakarta: Ministry of Education and Culture.

Anderson & Krathwohl. (2001). A Taxonomy for Learning, Teaching and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. New York: Addison Wesley Longman, Inc

Andrich, D. (1981). Book Review : Probabilistic Models for Some Intelligence and Attainment Tests (expanded edition. Applied Psychological Measurement, 5(4), 545– 550. doi:10.1177/014662168100500413.

Bond, TG & Fox, CM (2015). Applying the Rasch Model, Fundamental Measurement in the Human Science (3rd edition). New York: Routledge.

Bonee, WJ, Staver, JR, & Yale, MS (2014). Rasch Analysis in the Human Science. Dordrecht: Springer

Brogden, HE (1977). The rasch model, the law of comparative judgment and additive conjoint measurement. Psychometrics, 42(4), 631–634. doi:10.1007/bf02295985.

Brookhart, S. M. (2010). Higher-Order Thinking In Your Classroom. ACDC, Alexandria, Virginia USA.

Heong, Y. et.al. (2011). The level of marzano higher order thinking skills among technical education students. International Journal of Social Science and Humanity, 1(2), 121.

Karlimah, K., Andriani, D., & Suryana, D. (2020). Development of Mathematical Anxiety Instruments with a Rasch Model Analysis. The Open Psychology Journal, 13(1).

Linacre, JM (1999). Investigating Rating Scale Category Utility. Journal of Outcome Measurement. 3(2), 103-122.

Mardapi, D. (2017). Educational Measurement, Assessment, and Evaluation. Yogyakarta: Parama Publishing

Masters, GN (1982). A Rasch Model For Partial Credit Scoring. Psychometrics, 47(2), 149–174. doi:10.1007/bf02296272.

Misbach, IH and Sumintono, B. (2014) Development and validation of the instrument "student perceptions of the teacher's moral character" in Indonesia using the Rasch Model. In: National Psychometric Seminar: "Development of Valid Character Assessment Instruments", 24 May 2014, Solo, Indonesia.

Novinda, MRR, Silitonga, HTM, & Hamdani, H. (2019). Development of Multiple Choice Test Using Rasch Model for Class X Pontianak. Journal of Equatorial Education and Learning, 8(6).

Nur, L., Conscience, LA, Suryana, D., & Ahmad, A. (2020). Rasch model application on character development instrument for elementary school students. International Journal of Learning, Teaching and Educational Research, 19(3), 437-459.

Pratama, D. (2020). Quality Analysis of Teacher-Made Tests Through Rasch's Item Response Theory (IRT) Approach. Tarbawy: Journal of Islamic Education, 7(1), 61-70.

Sadikin, A. (2018). Evaluation and Process Biology learning outcomes. Jambi: Salim Media Indonesia.

Saftihry, EA (2019). Technical Assessment of Tests and Nontests. Malang: CV. IRDH.

Smith, RM (1986). Person Fit in the Rasch Model. Educational and Psychological Measurement, 46(2), 359–372. doi:10.1177/001316448604600210.

Sumintono, B. (2018). Rasch Model Measurements as Tools in Assessment for Learning. In 1st International Conference on Education Innovation (ICEI 2017). Atlantis Press.

Sumintono, B., & Widhiarso, W. (2015). RASCH Modeling Application in Educational Assessment. In TrimKom Publishing Home.

Tilaar, ALF (2018). Mathematics Learning Assessment. Tondano: Unima Press.

Zainul, A. (2001). Alternative assessment. Jakarta: Director General of Higher Education.

DOI: https://doi.org/10.17509/ijpe.v4i2.29318


  • There are currently no refbacks.

Copyright (c) 2020 Indonesian Journal of Primary Education

This Journal is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. 
©Indonesian Journal of Primary Education. ISSN: 2597-4866 (Online) dan 2599-2821 (Print).

Jalan Dadaha Nomor 18 Kota Tasikmalaya

Telepon (0265) 331860

Homepage http://pgsd-tasikmalaya.upi.edu/

Email pgsd_tasik@upi.edu