Development of a Scientific Literacy Assessment Instrument Based on the PISA 2018 Assessment and Analytical Framework at the Elementary Education Level

Hana Yunansah; Muslim Muslim; Ridwan Efendi; Raden Giovanni Ariantara

doi:10.17509/jds.v3i2.104849

Development of a Scientific Literacy Assessment Instrument Based on the PISA 2018 Assessment and Analytical Framework at the Elementary Education Level

Hana Yunansah, Muslim Muslim, Ridwan Efendi, Raden Giovanni Ariantara

Abstract

he low average achievement of Indonesian students' scientific literacy on PISA (Program for International Student Assessment) is thought to be because Indonesian students are not used to solving questions that test reasoning. The facts in the field show that the teacher has not specifically trained students' scientific literacy skills in science learning and the teacher's lack of ability in making assessment instruments to measure students' scientific literacy skills. This indicates the need to develop a scientific literacy assessment instrument that refers to the PISA framework. This study aims to develop a scientific literacy assessment instrument based on the PISA framework on science content in junior high schools and determine the achievement of students' scientific literacy skills. The research method used was R and D research. The instrument in this study was a scientific literacy test item developed based on the 2018 PISA framework which combines aspects of context, knowledge and competence. The simulation-based test and the three-dimensional learning framework generally satisfy logical validity, subject to certain considerations for future test development. The test also demonstrates empirical validity and reliability based on the results of extensive student testing. Therefore, it can be concluded that this developed product is capable of measuring students' cognitive abilities regarding the topic of light waves in accordance with the three-dimensional learning framework.

Keywords

Assessment instrument; PISA 2018 framework; Scientific literacy

Full Text:

PDF

References

Adi, N. R. M., Amaruddin, H., Adi, H. M. M., and A’yun, I. L. Q. (2022). Validity and reliability analysis using the rasch model to measure the quality of mathematics test items of vocational high schools. Journal of research and educational research evaluation.

Alfarisa, F., Arisetyawan, A., Maulana, F. R., Lathifah, N. Y., Afriani, S., and Mashudi, M. (2025). Creative thinking in elementary education: Reliable assessment instruments for science and social studies. Jurnal Penelitian Dan Evaluasi Pendidikan, 29(1), 13–25.

AlKaabi, N. A., Al-Maadeed, N., Romanowski, M. H., and Sellami, A. (2024). Drawing lessons from PISA: Qatar’s use of PISA results. PROSPECTS, 54(1), 221–240.

Alotaibi, W. H., and Alghamdi, A. K. H. (2025). Saudi secondary school science textbooks’ ability to inculcate a PISA-Informed Scientific Identity. PLOS One, 20(10).

Apriana, S., Lubis, A. A., and Sofyan, D. (2022). Higher-order thinking skills on teachers-made tests by english teachers of a senior high school in Bengkulu city. Journal of English Teaching, Applied Linguistics and Literatures (JETALL), 5(2), 137–152.

Arbeni, W., Windiani, A., Sihotang, D. S. B., Anggraini, N., Wulandari, S., and Nugroho, A. (2025). Test reliability analysis in educational evaluation: A quantitative approach to consistency and validity. Holistic Science, 5(1), 59–64.

Atikah, A., Sudiyatno, S., Rahim, A., and Marlina, M. (2022a). Assessing the item of final assessment mathematics test of junior high school using Rasch model. Jurnal Elemen, 8(1), 117–130.

Atikah, A., Sudiyatno, S., Rahim, A., and Marlina, M. (2022b). Assessing the item of final assessment mathematics test of junior high school using Rasch model. Jurnal Elemen, 8(1), 117–130.

De Vet, H. C. W., Mokkink, L. B., Mosmuller, D. G., and Terwee, C. B. (2017). Spearman–Brown prophecy formula and Cronbach’s alpha: Different faces of reliability and opportunities for new applications. Journal of Clinical Epidemiology, 85, 45–49.

Dewi Suprihatin, E. A. (2021). Scientific approach to promote scientific writing skills using blended learning system. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 12(13), 762–769.

Eidin, E., and Shwartz, Y. (2023). From Ideal to Practical—A Design of Teacher Professional Development on Socioscientific Issues. Sustainability, 15(14), 11394.

Ernawati, M. D. W., Asrial, A., Perdana, R., Septi, S. E., and Rahmi, R. (2021). Evaluation of Students’ Attitudes and Science Process Skills toward Middle School Science Subject in Indonesia. Jurnal Pendidikan Progresif, 11(2), 258–274.

Evita, E., Sari, E. D. K., Susieni, E., and Tamam, B. (2023). Analysis of item quality in islamic religious education subjects with rasch modeling. Conciencia, 23(2), 147–156.

Fattikasari, D. W., and Nasrudin, H. (2023). Development of electronic worksheets based on project-based learning assisted by a virtual laboratory to train students’ critical thinking skills. Jurnal Pijar Mipa, 18(4), 493–501.

Harris, L. R., Adie, L., and Wyatt-Smith, C. (2022). Learning progression–based assessments: a systematic review of student and teacher uses. Review of Educational Research, 92(6), 996–1040.

Khery, Y., Sarjan, M., Ahzan, S., and Efendi, I. (2022). Konseptualisasi literasi sains mengacu pada kerangka sains pisa sejak tahun 2000. Educatoria: Jurnal Ilmiah Ilmu Pendidikan, 2(4), 200–231.

Mallinson, T., Kozlowski, A. J., Johnston, M. V., Weaver, J., Terhorst, L., Grampurohit, N., Juengst, S., Ehrlich-Jones, L., Heinemann, A. W., Melvin, J., Sood, P., and Van De Winckel, A. (2022). Rasch reporting guideline for rehabilitation research (RULER): The RULER Statement. Archives of Physical Medicine and Rehabilitation, 103(7), 1477–1486.

Jari, M. (2022). Seeking the real item difficulty: Bias-corrected item difficulty and some consequences in Rasch and IRT modeling. Behaviormetrika, 50(1), 121–154.

Mulyanti, S., Sukmawati, W., and Tarkin, N. E. H. (2022). Development of items in acid-base identification experiments using natural materials: validity test with rasch model analysis. Phenomenon: Jurnal Pendidikan MIPA, 12(1), 17–30.

Nefianthi, R., and Dwinanda Wahab, M. N. N. (2024). The analysis of last year junior high school students’ scientific literacy in biology topics. BIO-INOVED: Jurnal Biologi-Inovasi Pendidikan, 6(3), 397.

Pokropek, A., Marks, G. N., Borgonovi, F., Koc, P., and Greiff, S. (2022a). General or specific abilities? Evidence from 33 countries participating in the PISA assessments. Intelligence, 92, 101653.

Pokropek, A., Marks, G. N., Borgonovi, F., Koc, P., and Greiff, S. (2022b). General or specific abilities? Evidence from 33 countries participating in the PISA assessments. Intelligence, 92, 101653.

Prastiwi, C. H. W., Rukmini, D., Saleh, M., and Astuti, P. (2024). Framework for assessing 21st century skills in EFL project-based learning. Proceedings of UNNES-TEFLIN National Conference, 6, 328–337.

Ramadhan, A. N., Dwiningrum, S. I. A., dan Subhan, B. (2021). Potensi integrasi pembelajaran biologi dengan pembelajaran quran-hadis. AL QUDS: Jurnal Studi Alquran Dan Hadis, 5(1), 263.

Schellekens, L. H., Bok, H. G. J., De Jong, L. H., Van Der Schaaf, M. F., Kremer, W. D. J., and Van Der Vleuten, C. P. M. (2021). A scoping review on the notions of Assessment as Learning (AaL), Assessment for Learning (AfL), and Assessment of Learning (AoL). Studies in Educational Evaluation, 71, 101094.

Sitanggang, Y. J., Simorangkir, T., Sihaloho, A. L., Pakpahan, R. S. M., and Silalahi, D. E. (2025). Analysis of english skill tests for grade vii students based on language assessment principles. JURNAL PENDIDIKAN BAHASA, 15(3), 349–360.

Soeharto, S., and Csapó, B. (2022). Assessing Indonesian student inductive reasoning: Rasch analysis. Thinking Skills and Creativity, 46, 101132.

Suherman, S., and Vidákovich, T. (2025). An assessment of creativity style inventory: A Rasch model. Journal of Creativity, 35(3), 100110.

Sukma, Abbas, A., Nurhayati, Kaharuddin, and Gheisari, A. (2022). Development of authentic assessment in local wisdom-based reading learning. Education Research International, 2022, 1–11.

Sweeney, S. M., Sinharay, S., Johnson, M. S., and Steinhauer, E. W. (2022). An investigation of the nature and consequence of the relationship between irt difficulty and discrimination. Educational Measurement: Issues and Practice, 41(4), 50–67.

Syah, R., Darmawan, D., and Syamsuddin, S. (2023). Development of a scientific attitude measurement instrument for Dharmacariya department students at STABN using the Fishbein and Ajzen model. Jurnal Penelitian Dan Evaluasi Pendidikan, 27(2), 226–240.

Yuniarto, B., Yudha, R. P., and Nurchotimah, A. S. I. (2022). Validity and reliability: Social attitude assessment instruments. Communications in Humanities and Social Sciences, 2(1), 1–6.

Zhang, L., Liu, X., and Feng, H. (2023). Development and validation of an instrument for assessing scientific literacy from junior to senior high school. Disciplinary and Interdisciplinary Science Education Research, 5(1), 21.

Zuhrotul Wardi, L., Wasis, W., Widodo, W., Satriawan, M., and Nisa, K.(2024). Trends and development in digital learning research for enhancing scientific literacy in physics learning: literature review and bibliometric analysis. Jurnal Pendidikan MIPA, 25(3), 1051–1066.

DOI: https://doi.org/10.17509/jds.v3i2.104849

Refbacks

There are currently no refbacks.

Journal of Didactic Studies https://ejournal.upi.edu/index.php/jds is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License

All rights reserverd. p-ISSN 2987-856X e-ISSN 2987-7512

Primary Teacher Education and Early Childhood Education study program
Universitas Pendidikan Indonesia, Cibiru Campus

Secretariat:
Main Building of Journal Publication Center, 2nd Floor
Jl. Raya Cibiru KM 15 Bandung-West Java, 40393

Username
Password
Remember me