Assessing and validating young Kazakhstanis’ reading skills in English, the impact of classroom climate, and their engagement on reading skills

Aigul Akhmetova, Soeharto Soeharto, Gaysha Imambayeva, Benő Csapó

Abstract


Improving reading skills in English for young Kazakhstani learners is a challenging process, but it was unfortunate that teachers in Kazakhstan do not frequently apply and implement assessments in the context of a school or classroom.  Following a pilot study conducted in 2018, a continuous assessment was considered necessary to highlight the errors and challenges related to such unfortunate condition. This study performed validation and assessment of reading skills in English as a foreign language (EFL) within a classroom climate and in relation to the engagement of students in grades 6 and 8 in Kazakhstan. The participants were chosen randomly from seven secondary schools in a major city. The first language of all the participants (N = 1,206) was either Kazakh or Russian. Data from 906 students following data screening were analyzed. EFL Reading comprehension tests and a questionnaire regarding the students’ classroom climate and engagement were administered via the eDia online assessment platform. The analysis involved an exploratory factor analysis, a confirmatory factor analysis, and an internal consistency test using Cronbach’s alpha, composite reliability, and construct validity. To determine the effects of factors detected from the background variables on students’ reading results in EFL, a regression analysis was conducted. As results, no gender differences were found for either grade. However, in grade 6, students whose first language was Kazakh performed better than the students whose first language and/or language spoken at home was Russian. In grade 8, the differences between Kazakh and Russian students were negative but non-significant. Nevertheless, despite the weak relationship between the latent factors and reading skills, both grades showed a good model fit to the data and good factor loadings. Implication of the study and further research are also discussed.


Keywords


Classroom climate; EFL; engagement; reading skills; validation

Full Text:

PDF

References


Akhmetova, A. B., & Csapó, B. (2018). Development of reading skills of 6th and 8th graders in English, Kazakh, and Russian from the perspective of young learners’ backgrounds in Pavlodar. In F. Anikó, S. Krisztián, & M. Helga (Eds.), XVIII National Conference on Education (pp. 363-363). http://onk2018.elte.hu/wp-content/uploads/2018/11/Absztrakt-k%C3%B6tet_FINAL.pdf

Alsowat, H. (2016). An EFL flipped classroom teaching model: Effects on English language higher-order thinking skills, student engagement and satisfaction. Journal of Education and Practice, 7(9), 108–121.

Baker, J. A. (2006). Contributions of teacher–child relationships to positive school adjustment during elementary school. Journal of School Psychology, 44(3), 211–229. https://doi.org/10.1016/j.jsp.2006.02.002

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: principles, policy & practice, 5(1), 7–74. https://doi.org/10.1080/0969595980050102

Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). Guilford publications.

Cline, T., & Shamsi, T. (2000). Language needs or special needs? The assessment of learning difficulties in literacy among children learning English as an additional language: A literature review. Department for Education and Skills. HMSO.

Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment. Cambridge University Press.

Cohen, L., Manion, L., & Morrison, K. (2007). Research methods in education (6th ed.). Routledge.

Csapó, B., & Nikolov, M. (2009). The cognitive contribution to the development of proficiency in a foreign language. Learning and Individual Differences, 19(2), 209–218. https://doi.org/10.1016/j.lindif.2009.01.002

Csapó, B., & Molnár, G. (2019). Online diagnostic assessment in support of personalized teaching and learning: The eDia system. Frontiers in Psychology, 10(1522). https://doi.org/10.3389/fpsyg.2019.01522

Davison, C., & Leung, C. (2009). Current issues in English language teacher‐based assessment. TESOL Quarterly, 43(3), 393–415. https://doi.org/10.1002/j.1545-7249.2009.tb00242.x

Dotterer, A. M., & Lowe, K. (2011). Classroom context, school engagement, and academic achievement in early adolescence. Journal of Youth and Adolescence, 40(12), 1649–1660. https://doi.org/10.1007/s10964-011-9647-5

Downer, J. T., Rimm-Kaufman, S. E., & Pianta, R. C. (2007). How do classroom conditions and children's risk for school problems contribute to children's behavioral engagement in learning? School Psychology Review, 36(3), 413–432. https://doi.org/10.1080/02796015.2007.12087938

Djigic, G., & Stojiljkovic, S. (2011). Classroom management styles, classroom climate and school achievement. In Z. Bekirogullari (Ed.), The 2nd International Conference on Education and Educational Psychology 2011 (pp. 819-828). Elsevier. https://doi.org/10.1016/j.sbspro.2011.11.310

Farrell, A. M. (2010). Insufficient discriminant validity: A comment on Bove, Pervan, Beatty, and Shiu (2009). Journal of business research, 63(3), 324–327. https://doi.org/10.1016/j.jbusres.2009.05.003

Field, A. (2009). Discovering Statistics Using SPSS (3rd ed.) Sage Publications.

Fletcher, J. (2015). What can we do to support reading for young adolescent students? Education 3-13, 45(2), 258–271. https://doi.org/10.1080/03004279.2015.1078832

Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of educational research, 74(1), 59–109. https://doi.org/10.3102/00346543074001059

Furrer, C., & Skinner, E. (2003). Sense of relatedness as a factor in children's academic engagement and performance. Journal of Educational Psychology, 95(1), 148–162. https://doi.org/10.1037/0022-0663.95.1.148

Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (2010). Multivariate data analysis with readings. Prentice Hall.

Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135. https://doi.org/10.1007/s11747-014-0403-8

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

Jia, Y., Way, N., Ling, G., Yoshikawa, H., Chen, X., Hughes, D., Ke, X., & Lu, Z. (2009). The influence of student perceptions of school climate on socioemotional and academic adjustment: A comparison of Chinese and American adolescents. Child Development, 80(5), 1514–1530. https://doi.org/10.1111/j.1467-8624.2009.01348.x

Johnson, K. (2008). An introduction to foreign language learning and teaching (2nd ed.). Routledge. https://doi.org/10.4324/9781315834603

Kock, N. (2014). Advanced mediating effects tests, multi-group analyses, and measurement model assessments in PLS-based SEM. International Journal of e-Collaboration, 10(1), 1–13. https://doi.org/10.4018/ijec.2014010101

Lee, J., & Schallert, D. L. (2014). Literate actions, reading attitudes, and reading achievement: Interconnections across languages for adolescent learners of English in Korea. The Modern Language Journal, 98(2), 553–573. https://doi.org/10.1111/modl.12088

Myrberg, E., & Rosén, M. (2009). Direct and indirect effects of parents' education on reading achievement among third graders in Sweden. British Journal of Educational Psychology, 79(4), 695–711. https://doi.org/10.1348/000709909X453031

Nikolov, M. (2016). A framework for young EFL learners’ diagnostic assessment: ‘Can do statements’ and task types. In M. Nikolov (Ed.), Assessing young learners of English: Global and local perspectives (pp. 65-92). Springer. https://doi.org/10.1007/978-3-319-22422-0_4

Nikolov, M. (2017). Students’ and teachers’ feedback on diagnostic tests for young EFL learners: Implications for classrooms. In M. P. García Mayo (Ed.), Learning foreign languages in primary school: Research insights (pp. 249–266). Multilingual Matters. https://doi.org/10.21832/9781783098118-014

Nikolov, M., & Csapó, B. (2010). The relationship between reading skills in early English as a foreign language and Hungarian as a first language. International Journal of Bilingualism, 14(3), 315–329. https://doi.org/10.1177/1367006910367854

Nikolov, M., & Csapó, B. (2018). The relationships between 8th graders’ L1 and L2 reading skills, inductive reasoning and socio-economic status in early English and German as a foreign language programs. System, 73, 48–57. https://doi.org/10.1016/j.system.2017.11.001

OECD (2014), Reviews of national policies for education: Secondary education in Kazakhstan. OECD Publishing. http://doi.org/10.1787/9789264205208-en

OECD & The World Bank (2015), OECD reviews of school resources: Kazakhstan 2015. OECD Publishing. http://dx.doi.org/10.1787/9789264245891-en

OECD (2017). PISA 2015 Assessment and analytical framework: Science, reading, mathematics and financial literacy. OECD publishing.

OECD (2018). PISA for development assessment and analytical framework: Reading, mathematics and science. OECD Publishing. http://doi.org/10.1787/9789264305274-en

Onwuegbuzie, A. J., Witcher, A. E., Collins, K. M., Filer, J. D., Wiedmaier, C. D., & Moore, C. W. (2007). Students’ perceptions of characteristics of effective college teachers: A validity study of a teaching evaluation form using a mixed-methods analysis. American Educational Research Journal, 44(1), 113–160. https://doi.org/10.3102/0002831206298169

Perliger, A., Canetti-Nisim, D., & Pedahzur, A. (2006). Democratic attitudes among high-school pupils: The role played by perceptions of class climate. School Effectiveness and School Improvement, 17(1), 119–140. https://doi.org/10.1080/09243450500405217

Pianta, R. C., Belsky, J., Vandergrift, N., Houts, R., & Morrison, F. J. (2008). Classroom effects on children’s achievement trajectories in elementary school. American Educational Research Journal, 45(2), 365–397. https://doi.org/10.3102/0002831207308230

Pianta, R.C., Hamre, B.K., & Allen, J.P. (2012). Teacher-student relationships and engagement: conceptualizing, measuring, and improving the capacity of classroom interactions. In S. Christenson, A. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 365 –386). Springer. https://doi.org/10.1007/978-1-4614-2018-7_17

Podsakoff, P. M., MacKenzie, S. B., Lee, J. Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. https://doi.org/10.1037/0021-9010.88.5.879

Pons, A., Amoroso, J., Herczynski, J., Kheyfets, I., Lockheed, M., & Santiago, P. (2015). OECD reviews of school resources. OECD Publishing.

Reyes, M. R., Brackett, M. A., Rivers, S. E., White, M., & Salovey, P. (2012). Classroom emotional climate, student engagement, and academic achievement. Journal of Educational Psychology, 104(3), 700–712. https://doi.org/10.1037/a0027268

Rixon, S. (2016). Do developments in assessment represent the ‘coming of age’ of young learners English language teaching initiatives? The international picture. In M. Nikolov (Eds.), Assessing young learners of English: Global and local perspectives (pp. 19-41). Springer. https://doi.org/10.1007/978-3-319-22422-0_2

Roever, C., & McNamara, T. (2006). Language testing: The social dimension. International Journal of Applied Linguistics, 16(2), 242–258. https://doi.org/10.1111/j.1473-4192.2006.00117.x

Ryan, A., & Patrick, H. (2001). The classroom social environment and changes in adolescents’ motivation and engagement during middle school. American Educational Research Journal, 38, 437–460. https://doi.org/10.3102/00028312038002437

Sagintayeva, A., Bridges, D., McLaughlin, C., Mehisto, P., Drummond, M.J., Ayubayeva, N., Kishkentayeva, M., Kulakhmetova, A., Sadvakassova, M., Gasskov, V., Ganimurat, N., Canning, M., Finney, J.E., Jones, D.P., Mcguinness, A.C., Harvey, D., Bilyalov, D., & Sagyndykova, Z. (2014). Development of strategic directions for education reforms in Kazakhstan for 2015-2020, Diagnostic report. Indigo print.

Sembok, T. M., Zaman, H. B., & Kadir, R. A. (2008). IRQAS: Information retrieval and question answering system based on a unified logical-linguistic model. In L. Kazovsky, P. Borne, N. Mastorakis, A. Kuri-Morales, I. Sakellaris (Eds.), Proceedings of the 7th WSEAS international conference on artificial intelligence, knowledge engineering and data bases (pp. 460-464). World Scientific and Engineering Academy and Society (WSEAS). https://dl.acm.org/doi/10.5555/1415881.1415954

Singleton, C., & Vincent, D. (2004). Assessing literacy: Current challenges and issues. Journal of Research in Reading, 27(2), 113–117. https://doi.org/10.1111/j.1467-9817.2004.00219.x

Stanovich, K. E. (2000). Progress in understanding reading: Scientific foundations and new frontiers. Guilford Press.

Thompson, B. (2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. American Psychological Association.

Wang, M. T., & Eccles, J. S. (2011). Adolescent behavioral, emotional, and cognitive engagement trajectories in school and their differential relations to educational success. Journal of Research on Adolescence, 22(1), 31–39. https://doi.org/10.1111/j.1532-7795.2011.00753.x

Way, N., Reddy, R., & Rhodes, J. (2007). Students’ perceptions of school climate during the middle school years: Associations with trajectories of psychological and behavioral adjustment. American journal of community psychology, 40(3-4), 194–213. https://doi.org/10.1007/s10464-007-9143-y

Willms, J. D., Friesen, S. & Milton, P. (2009). What did you do in school today? Transforming classrooms through social, academic, and intellectual engagement. Canadian Education Association.

Zimmerman, B. J. (2000). Attaining self-regulation: A social cognitive perspective. In M. Boekaerts, P.R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 13-39). Academic Press. https://doi.org/10.1016/B978-012109890-2/50031-7




DOI: https://doi.org/10.17509/ijal.v12i2.37321

Refbacks



View My Stats

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.