HOTS-based language assessment literacy: Challenges and prospects in English language teaching

Sumardi Sumardi, Rizki Indra Guci

Abstract


The issues of inaccurate and invalid language assessment were mainly due to teachers’ lower level of assessment literacy and teachers' inadequate skills to design higher-order thinking skills (henceforth HOTS) based language assessments. Thus, this paper aims to mainly portray teachers’ HOTS-based language assessment literacy in English language teaching (ELT) classrooms after they participated in training on HOTS-based language assessment. A case study was employed, and forty-three ELT teachers from secondary schools voluntarily participated in the current study. Some of them were interviewed, and the test items they constructed were analyzed to examine if their HOTS-based language assessment literacy was reflected in the quality of the test items. The findings showed that ELT teachers who engaged in training gained the basic level of knowledge and comprehension of HOTS-based language assessment. However, their knowledge and comprehension are not parallel with their skills in constructing HOTS-based test items. The paper ends with recommendations for the stakeholders (school principals, educational authorities, and training providers) responsible for designing future training programs to enhance ELT teachers' language assessment literacy.

Keywords


ELT teachers; HOTS; language assessment literacy; need analysis; training programs

Full Text:

PDF

References


Ary, D., Jacobs, L.C & Sorensen, C. 2010. Introduction to research in education (8th ed.). Wadsworth

Assalahi, H. (2021). The professional standards for teachers in Saudi Arabia as a framework for EFL teachers’ professional development needs assessment. TESOL International Journal, 16(6.1), 104 – 117. http://doi.org/10.7575/aiac.alls.v.12n.6.p.13

Battistone, W., Buckmiller, T., & Peters, R. (2019). Assessing assessment literacy: are new teachers prepared to assume jobs in school districts engaging in grading and assessment reform efforts? Studies in Educational Evaluation, 62, 10 – 17. https://doi.org/10.1016/j.stueduc.2019.04.009

Bohn, H. (2018). Assessing content in a curriculum-based ELT oral exam: the importance of higher-order thinking skills. Journal of Language Teaching and Research, 9(1), 16-26. https://doi.org/10.17507/jltr.0901.03

Brookhart, S. M. (2010). How to assess higher-order thinking skills in your classroom. SCD Alexandria.

Budiman, A., & Jailani. (2015, May 17 - 19). Developing an assessment instrument of higher-order thinking skill (HOTS) in mathematics for junior high school grade VIII semester 1 (Conference Session). http://eprints.uny.ac.id/22986/

Bybee, R. W. (1997). Achieving scientific literacy: From purposes to practices. Heinemann.

Cheng, L. & Watanabe, Y. (2008). Washback in language testing. Taylor & Francis.

Coombe, C., S. Troudi & Al-Hamly, M. (2012). Foreign and second language teacher assessment literacy: issues, challenges, and recommendations. In C. Coombe, P. Davidson, B. O’Sullivan, & S. Stoynoff (Eds.), The Cambridge guide to second language assessment (pp. 20-29). Cambridge University Press.

Davies, A. (2008). Textbook trends in teaching language testing. Language Testing, 25(3), 327–347. https://doi.org/10.1177/0265532208090156

Fulcher, G. (2012). Assessment literacy for the language classroom. Language Assessment Quarterly, 9(2), 113-132. https://doi.org/10.1080/15434303.2011.642041

Gotch, C. M., & McLean, C. (2019). Teacher outcomes from a statewide initiative to build assessment literacy. Studies in Educational Evaluation, 62, 30–36. https://doi.org/10.1016/j.stueduc.2019.04.003

Hasselgreen, A., Carlsen, C. & Helness, H. (2004, April 11). European survey of language and assessment needs. Part one: general findings. http://www.ealta.eu.org/documents/resources/survey-report-pt1.pdf

Heong, Y. M., Yunos, J. M, Othman, W., Hassana, R., Kionga, T.T., & Mohamad, M. M. (2012). The needs analysis of learning higher-order thinking skills for generating ideas. Procedia - Social and Behavioral Sciences, 59, 197 – 203. https://doi.org/10.1016/j.sbspro.2012.09.265

Inbar-Lourie, O. (2008). Constructing a language assessment knowledge base: A focus on language assessment courses. Language Testing, 25(3), 385 - 402. https://doi.org/10.1177/0265532208090158

Ioannou-Georgiou, S., & Pavlou, P. (2003). Assessing young learners. Oxford University Press.

Jansen, T., & Moller, J. (2022). Teacher judgments in school exams: Influences of students' lower order-thinking skills on the assessment of students’ higher-order thinking skills. Teaching and Teacher Education, 111, 1 – 10. https://doi.org/10.1016/j.tate.2021.103616

Kaiser, G., & Willander, T. (2005). Construction of mathematical literacy: Results of an empirical study. Teaching Mathematics and Its Applications, 24(2), 48 – 60. https://doi.org/10.1093/teamat/hri016

Kubiszyn, T. & Borich, G. D. (2013). Educational testing & measurement: classroom application and practice (10th ed.). John Wiley & Sons, Inc.

Kvasova, O. & Kavytska, T. (2014). The assessment competence of University Foreign Language Teachers: A Ukrainian perspective. Language Learning in Higher Education, 4(1), 159-177. https://doi.org/10.1515/circles-2014-0010

Lam, R. (2014). Language assessment training in Hong Kong: implications for language assessment literacy. Language Testing, 32(2), 169 - 197. https://doi.org/10.1177/0265532214554321

Lan, C., & Fan, S. (2019). Developing classroom-based language assessment literacy for in-service ELT teachers: The gaps. Studies in Educational Evaluation, 61, 112 – 122. https://doi.org/10.1016/j.stueduc.2019.03.003

Mansouri, B., Molana, K., & Nazari, M. (2021). The interconnection between second language teachers’ language assessment literacy and professional agency: The mediating role of institutional policies. System, 103, 1 – 11. https://doi.org/10.1016/j.system.2021.102674

McMillan, J. H. (2008). Educational research: fundamentals for the consumer (4th ed.). Pearson Education, Inc.

Miles, M. B., Huberman, A. M., & Saldana, J. (2014). Qualitative data analysis (3rd ed.). Sage Publication.

Moeiniasl, M. Taylor, L., deBraga, M., Manchanda, T., Huggon, T., & Graham, J. (2022). Assessing the critical thinking skills of English language learners in a first year psychology course. Thinking Skills and Creativity, 43, 1 – 13. https://doi.org/10.1016/j.tsc.2022.101004

Mohamed, R., & Lebar, O. (2017). Authentic assessment in assessing higher-order thinking skills. International Journal of Academic Research in Business and Social Sciences, 2, 466 – 476. https://doi.org/10.6007/IJARBSS/v7-i2/2021

Pfingsthorn, J., & Weltgen, J. (2022). Inclusive and fair assessment in foreign language education: The role of fundamental attribution error in the evaluation of students’ performance. International Journal of Educational Research, 3, 1 – 11. https://doi.org/10.1016/j.ijedro.2022.100160

Pill, J., & Harding, L. (2013). Defining the language assessment literacy gap: Evidence from a parliamentary inquiry. Language Testing, 30(3), 381–402. https://doi.org/10.1177/0265532213480337

Ramasamy, S., Rahman, F.A., Ismail, H., Manaf, U.K.A., & Said, R.R. (2016). Teachers' levels of knowledge and interest in higher-order thinking skills (HOTS) according to the field taught and category of schools. Journal of Modern Education Review, 6(9), 611 – 621. https://doi.org/10.15341/jmer(2155-7993)/09.06.2016/005

Rea-Dickins, P. (2000). Assessment in early years language learning contexts. Language Testing, 17(2), 115–122. https://doi.org/10.1177/026553220001700201

Rea-Dickins, P. (2001). Mirror, mirror on the wall: Identifying processes of classroom assessment. Language Testing, 18, 429–462. https://doi.org/10.1177/026553220101800407

Rea‐Dickins, P. (2008). Classroom‐Based Language Assessment. In Hornberger, N. H., Encyclopedia of Language and Education. Springer. https://doi.org/10.1007/978-0-387-30424-3_180

Retnowati, H., Djidu, H., Kartianom, K., Apino, E., & Anazifa, R.D. (2018). Teachers’ knowledge about Higher-order thinking skills and its learning strategy. Problems of Education in the 21st Century, 77(2), 215 – 230. https://doi.org/10.33225/pec/18.76.215

Roslan, R., Nishio, Y., & Jawawi, R. (2022). Analyzing English language teacher candidates’ assessment literacy: A case of Bruneian and Japanese universities. System, 111, 1 – 9. https://doi.org/10.1016/j.system.2022.102934

Sah, R. (2021). Students as partners in learning and teaching: Assessing the effectiveness of student evaluation of teaching. Asian EFL Journal, 28(3), 66-85.

Scarino, A. (2013). Language assessment literacy as self-awareness: understanding the role of interpretation in assessment and in teacher learning. Language Testing, 30(3), 309-327. https://doi.org/10.1177/0265532213480128

Singh, R. K. V., & Shaari, A.H. (2019). The analysis of higher-order thinking skills in English reading comprehension tests in Malaysia. Malaysian Journal of Society and Space, 15(1), 12 – 26. https://doi.org/10.17576/geo-2019-1501-02

Stiggins, R. J. (1991). Assessment Literacy. Phi Delta Kappa.

Stiggins, R. J. (2001). Evaluating classroom assessment training in teacher education programs. Educational Measurement: Issues and Practice, 18(1), 23–27. https://doi.org/10.1111/j.1745-3992.1999.tb00004.x

Stiggins, R. J. (2001). Student-involved classroom assessment (3rd ed.). Prentice-Hall.

Tian, W., Louw, S., & Khan, M.K. (2021). Covid-19 as a critical incident: Reflection on language assessment literacy and the need for radical changes. System, 103, 1 – 10. https://doi.org/10.1016/j.system.2021.102682

Truckenmiller, A. J., Cho, E., & Troia, G. A. (2022). Expanding assessment to instructionally relevant writing components in middle school. Journal of School Psychology, 94, 28 – 48. https://doi.org/10.1016/j.jsp.2022.07.002

Tsagari, D., & Vogt, K. (2017). Assessment literacy of foreign language teachers around Europe: research, challenges and future prospects. Papers in Language Testing and Assessment 6(1), 41 – 63.

Van de Walle, J. A. (2007). Elementary and middle school mathematics: teaching constructionally (6th ed.). Pearson Education, Inc.

Vogt, K., & Tsagari, D. (2014). Assessment literacy of foreign language teachers: findings of a European study. Language Assessment Quarterly, 11(4), 374-402, https://doi.org/10.1080/15434303.2014.960046

Widana, I. W. (2017). Higher-order thinking skills assessment (HOTS). Journal of Indonesian Assessment and Evaluation, 3(1), 32 – 44. https://doi.org/10.21009/JISAE.031.04

Yoke, S. K., Hasan, N. H., Jangga, R., & Kamal, S. N. M. (2015). Innovating with HOTS for the ESL reading class. English Language Teaching, 8(8), 10 – 17. http://dx.doi.org/10.5539/elt.v8n8p10

Yphantides, J. (2021). English foreign language teaching and professional identity development. Asian EFL Journal, 28(2.1), 5 – 18. http://doi.org/10.30479/JMRELS.2019.10271.1278

Yu, G. (2013). From integrative to integrated language assessment: are we there yet? Language Assessment Quarterly: An International Journal, 10(1), 110-114, http://dx.doi.org/10.1080/15434303.2013.766744




DOI: https://doi.org/10.17509/ijal.v12i3.44261

Refbacks

  • There are currently no refbacks.


View My Stats

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.