STEM Club Evaluation Scale: Validity and Reliability Study
Abstract
In STEMNET's report, 76% of 500 teachers interviewed stated that joining the STEM Club increased students' ability to solve real-world problems. This study aims to develop a valid and reliable measurement tool for evaluating STEM clubs. The research sample consisting of 149 teachers who carry out STEM club activities in schools in Turkey was determined using the purposive sampling method. Content and construct validity and reliability analyses have been performed for this purpose. To ensure content validity, (1) a pool of questions based on the literature was created, (2) draft scale items were determined, (3) an expert was allowed to check them, and (4)item difficulty and discrimination index were calculated. To ensure construct validity, (1) exploratory factor analyses (EFA) and (2) confirmatory factor analyses (CFA) were performed on both the same and different samples. As a result of the analyses, having the same data set be analyzed with different software was sufficient for verifying the factor structure. A three-factor structure consisting of 29 items was obtained, which explains 52% of the variance. Cronbach’s alpha of reliability for the overall scale was calculated as .92. As a result, a valid and reliable scale was determined to have been developed for researchers and program practitioners to evaluate STEM clubs. Suggestions have been made that the scale can be used on STEM clubs at the provincial, district, and school levels to determine their efficiency and productivity.
Full Text:
Download PDFReferences
Acar-Güvendir, M., & Özer-Özkan, Y. (2015). The examination of scale development and scale adaptation articles published in Turkish academic journals on education. Electronic Journal of Social Science, 14(52), 23-33. https://doi.org/10.17755/esosder.54872
Afterschool Alliance. (2015). Full STEM Ahead: Afterschool Programs Step Up as Key Partners in STEM Education. Washington, D.C. Retrieved from http://www.afterschoolalliance.org/AA3PM/.
Akgunduz, D., Aydeniz, M., Cakmakcı, G., Cavas, B., Corlu, M. S., Oner, T., & Ozdemir, S. (2015). STEM education Turkey report. Istanbul: Scala Press.
Akkus, A. (2019). Developing a Scale to Measure Students’ Attitudes toward Science. International Journal of Assessment Tools in Education, 6(4), 706-720. https://doi.org/10.21449/ijate.548516
Altunel, M. (2018). STEM education and Turkey: opportunities and risks. Seta Perspective, 207, 1-7.
Ayers, K. A., Wade-Jaimes, K., Wang, L., Pennella, R. A., & Pounds, S. B. (2020). The St. Jude STEM Clubs: An After-school STEM Club for Upper Elementary School Students in Memphis, TN. Journal of STEM outreach, 3(1), 1-26. https://doi.org/10.15695/jstem/v3i1.13
Baldridge, A., Nutt, A., Vaughn, M., Hartley-Lewis, C., & Amos, A. (2009). The STEM Club at Marietta High School. ASEE Southeast Section Conference.
Baran, E., Bilici, S. C., Mesutoglu, C., & Ocak, C. (2016). Moving STEM beyond schools: Students’ perceptions about an out-of-school STEM education program. International Journal of Education in Mathematics, Science and Technology, 4(1), 9-19.. http://dx.doi.org/10.18404/ijemst.71338
Beavers, A. S., Lounsbury, J. W., Richards, J. K., Huck, S. W., Skolits, G. J., & Esquivel, S. L. (2013). Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research, and Evaluation, 18(1), 6. https://doi.org/10.7275/qv2q-rk76
Bell, P., Lewenstein, B., Shouse, A.W., & Feder, M.A. (2009). Learning science in informal environments: People, places, and pursuits. Washington, DC: The National Academies Press.
Birturk, H. (2015). The functionality of social clubs in schools. Unpublished master's thesis. Yeditepe University Institute of Educational Sciences, Istanbul.
Blanchard, M. R., Hoyle, K. S., & Gutierrez, K. S. (2017). How to start a STEM club. Science Scope, 41(3), 88-94.
Burak, D., & Gultekin, M. (2021). Verbal-Visual Learning Styles Scale: Developing a Scale for Primary School Students. International Journal on Social and Education Sciences, 3(2), 287-303. https://doi.org/10.46328/ijonses.171
Buyruk, B., & Korkmaz, Ö. (2016). STEM Awareness Scale (SAS): Validity and Reliability Study. Journal of Turkish Science Education, 11(1), 3-23. https://doi.org/10.12973/tused.10179a
Buyukuzturk, S., Kilic-Cakmak, E., Akgun, O. E., Karadeniz, S. & Demirel, F. (2016). Scientific research methods (22nd Edition). Ankara: Pegem Academy.
Bybee, R.W. (2001). Achieving scientific literacy: Strategies for ensuring that free choice science education complements national formal science education efforts. In J.H. Falk (Ed.), Free choice education: How we learn science outside of school (pp. 44–63).
Cermik, H., & Kara, I. (2020). Physics course attitudes scale for high school students: a validity and reliability study. International Journal of Assessment Tools in Education, 7(1), 62-72. https://doi.org/10.21449/ijate.693211
Cevik, M. (2017). A study of STEM Awareness Scale development for high school teachers. Journal of Human Sciences, 14(3), 2436-2452. DOI:10.14687/jhs.v14i3.4673
Corlu, M. S., Capraro, R. M., & Capraro, M. M. (2014). Introducing STEM education: Implications for educating our teachers in the age of innovation. Education and Science, 39(171), 74-85. http://hdl.handle.net/11693/13203
Coskun, T. K., Alakurt, T., & Yilmaz, B. (2020). STEM Education from the Perspective of Information Technologies Teachers. Abant Izzet Baysal University Journal of the Faculty of Education, 20(2), 820-836. https://doi.org/10.17240/aibuefd.2020..-536856
Creswell, J. W. (2017). Research design qualitative, quantitative, and mixed-method studies (3rd edition) (S. B. Demir, Trans. Ed.). Ankara: Educating Book.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334.
Dabney, K. P., Tai, R. H., Almarode, J. T., Miller-Friedmann, J. L., Sonnert, G., Sadler, P. M., & Hazari, Z. (2012). Out-of-school time science activities and their association with a career interest in STEM. International Journal of Science Education, Part B, 2(1), 63-79. https://doi.org/10.1080/21548455.2011.629455
Derin, G., Aydin, E., & Kirkic, K. A. (2017). A Scale on the attitudes towards STEM education. El-Cezeri Journal of Science and Engineering, 4(3), 547-559. https://doi.org/10.31202/ecjse.336550
DeVellis, R. F. (2014). Scale development: Theory and applications. https://doi.org/10.14689/ejer.2016.63.2
Donmez, I. (2020). Adaptation of STEM Motivation Scale into Turkish: Validity and Reliability Study. YYU Journal of Education Faculty, 17(1), 486-510. https://doi.org/10.33711/yyuefd.693825
Ebel, R. L. & Frisbie, D. A. (1991). Essentials of educational measurement. Englewood Cliffs, NJ: Prentice-Hall.
Eroglu, S., & Bektas, O. (2016). Ideas of Science Teachers took STEM Education about STEM-based Activities. Journal of Qualitative Research in Education, 4(3), 43-67. https://doi.org/10.14689/issn.2148-2624.1.4c3s3m
Faber, M., Unfried, A., Wiebe, E. N., Corn, J., Townsend, L. W., & Collins, T. L. (2013, June). Student attitudes toward STEM: The development of the upper elementary school and middle/high school student surveys. In 2013 ASEE Annual Conference & Exposition (pp. 23-1094).
Ferrara, M., Mason, H., Wee, B., Rorrer, R., Jacobson, M., & Gallagher, D. (2017). Enriching undergraduate experiences with outreach in school STEM clubs.
Fidan, M., & Tuncel, M. (2021). Developing A Self-Efficacy Scale Toward Physics Subjects For Lower-Secondary School Students. Journal of Baltic Science Education, 20(1), 38. https://doi.org/10.33225/jbse/21.20.38
Firdaus, F., Subchan, W., & Narulita, E. (2020). Developing STEM-based TGT learning model to improve students' process skills. JPBI (Jurnal Pendidikan Biologi Indonesia), 6(3), 413-422. https://doi.org/10.22219/jpbi.v6i3.12249
Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education.
Fraenkel, J., & Wallen, N. (1996). Validity and reliability. How to design and research in education. New York: McGraw-Hill, INC, 3, 153-171.
Gabrielson, E. A., Strachan, J. L., Warner, M. T., & LaFleche, S. H. (2009). Evaluating the London Science Museum’s Activity Boxes at UK STEM Clubs.
Gelen, B., Akcay, B., Tiryaki, A., & Benek, I. (2019). Pre-Service Science Teachers’ Self-Efficacy toward Science, Technology, Engineering, Mathematics (STEM) Survey: An Adaptation to Turkish, Validity and Reliability Study. Journal of Theory and Practice in Education, 15(1), 88-107. https://doi.org/10.17244/eku.395204
George, D., & Mallery, P. (2016). IBM SPSS statistics 26 step by step: A simple guide and reference. (14th ed.). Routledge.
Gonsalves, A., Rahm, J., & Carvalho, A. (2013). “We could think of things that could be science”: Girls' re‐figuring of science in an out‐of‐school‐time club. Journal of Research in Science Teaching, 50(9), 1068-1097. https://doi.org/10.1002/tea.21105
Gottfried, M. A., & Williams, D. (2013). STEM club participation and STEM schooling outcomes. Education Policy Analysis Archives, 21, 79. DOI:10.14507/epaa.v21n79.2013
Guzey, S. S., Harwell, M., & Moore, T. (2014). Development of an instrument to assess attitudes toward science, technology, engineering, and mathematics (STEM). School Science and Mathematics, 114(6), 271-279. https://doi.org/10.1111/ssm.12077
Hacıomeroglu, G., & Bulut, A. S. (2016). Integrative STEM teaching intention questionnaire: a validity and reliability study of the Turkish form. Journal of Theory and Practice in Education, 12(3), 654-669. http://eku.comu.edu.tr/article/view/5000176286/5000164803
Hair, J. F., Black, W. C., Tatham, R. L. & Anderson, R. E. (2019). Multivariate data analysis (Eighth Edition). Cengage Learning EMEA.
Johnson, B., & Christensen, L. (2014). Educational research: Quantitative, qualitative and mixed approaches (Trans. Ed. Demir, S. B.). Ankara: Egiten Book.
Jöroskog, K., & Sörbom, D. (1993). Lisrel 8: structural equation modeling with the simplis command language. Lincolnwood: Scientific Software International, Inc.
Kalkan, C., & Eroglu, S. (2017). Designing sample activities based on STEM materials for gifted/talented students in support education rooms. Journal of Gifted Education and Creativity, 4(2), 36-46. https://dergipark.org.tr/tr/pub/jgedc/issue/38702/449432
Karadogan, S. (2016). In education-school learning practices and daily classroom problems. In Academic Evaluations and Solution Suggestions for Educational Problems in Turkey-1, (Ed. R Aksu), Publiser Maya Academy, 47-84.
Karakaplan, S., & Yildiz, H. (2010). A Study On Developing A Postpartum Comfort Questionnaire. Maltepe University Journal of Nursing Science and Art, 3(1), 55-65.
Kizilay, E., Yamak, H., & Kavak, N. (2019). Motivation Scale for STEM Fields. Journal of Computer and Education Research, 7(14), 540-557. https://doi.org/10.18009/jcer.617514
Lipuma, J., Bukiet, B. G., & Leon, C. (2021). Hands-on Developmental Playbook for STEM Clubs in Elementary Schools. STEM for Success Resources. 3. https://digitalcommons.njit.edu/stemresources/3
Mahoney, J. L., Parente, M. E., & Lord, H. (2007). After-school program engagement: Links to child competence and program quality and content. The Elementary School Journal, 107(4), 385-404.
Milner, D. I., Horan, J. J., & Tracey, T. J. (2014). Development and evaluation of STEM interest and self-efficacy tests. Journal of Career Assessment, 22(4), 642-653. https://doi.org/10.1177%2F1069072713515427
Ministry of National Education. (2016). STEM Education Report. http://yegitek.meb.gov.tr/STEM_Egitimi_Raporu.pdf, (Access date: 10 September 2021).
Ministry of National Education. (2018). Curriculum Monitoring and Evaluation System-Curriculums. http://mufredat.meb.gov.tr/Programlar.aspx, (Access date: 10 September 2021).
MoNE, (2017). MoNE Educational Institutions Social Activities Regulation.
National Research Council (2015). Identifying and supporting productive STEM programs in out-of-school settings. National Academies Press.
Oner, N. (2008). Examples of psychological tests used in Turkey: A reference (extended 2nd edition). Istanbul: Bogazici University Publishing.
Pallant, J. (2016). SPSS user guide Step-by-step data analysis with SPSS. (S. Balci & B. Ahi, Trans.). Ankara: Ani Publishing.
Pedaste, M., Baucal, A., & Reisenbuk, E. (2021). Towards a science inquiry test in primary education: development of items and scales. International Journal of STEM Education, 8(1), 1-19.
Polat, B. S. (2017). Investigation of Opinions of Administrators, Teachers and Students About the Effectiveness of Social Club Activities in Secondary Schools. Unpublished master's thesis. Ataturk University Institute of Education Sciences, Erzurum.
Qasem, M. A. N., & Gul, S. B. A. (2014). Effect of items direction (positive or negative) on the factorial construction and criterion-related validity in Likert scale. Khazar Journal of Humanities and Social Sciences, 17(3), 77-84. http://hdl.handle.net/20.500.12323/3240
Robelen, E. (2011). New STEM schools target underrepresented groups. Education Week, 31(1), 18-19.
Sahin, A. (2013). STEM clubs and science fair competitions: Effects on post-secondary matriculation. Journal of STEM Education: Innovations and Research, 14(1), 5-11.
Sahin, A., Ayar, M. C. & Adigüzel, T. (2014). STEM Related After-School Program Activities and Associated Outcomes on Student Learning. Educational Sciences: Theory and Practice, 14(1), 309-322. http://dx.doi.org/10.12738/estp.2014.1.1876
Secer, İ. (2017). Practical Data Analysis with Spss and Lisrel: Analysis and Reporting (3rd ed). Ankara: Ani Publishing.
Siew, N. M., Amir, N., & Chong, C. L. (2015). The perceptions of pre-service and in-service teachers regarding a project-based STEM approach to teaching science. SpringerPlus, 4(1), 8. http://dx.doi.org/10.1186/2193-1801-4-8
Straw, A. D., Branson, K., Neumann, T. R., & Dickinson, M. H. (2011). Multi-camera real-time three-dimensional tracking of multiple flying animals. Journal of The Royal Society Interface, 8(56), 395-409.
Tabachnick, B. G. & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston, MA: Allyn & Bacon/Pearson Education.
Tavsancil, E. (2002). Measuring attitudes and data analysis with SPSS. Ankara: Nobel Publishing.Turgut, M. F., & Baykul, Y. (2015). Measurement and evaluation in education (7th Edition). Ankara: PegemA Publishing.
TUSIAD (2014). A research on the demands and expectations of the workforce trained in STEM (Science, Technology, Engineering and Mathematics, Science, Technology, Engineering, Mathematics). TUSIAD.
Unlu, Z. K., Dokme, I., & Veli, U. N. L. U. (2016). Adaptation of the science, technology, engineering, and mathematics career interest survey (STEM-CIS) into Turkish. Eurasian Journal of Educational Research, 16(63), 21-36 http://dx.doi.org/ 10.14689/ejer.2016.63.2
Vural, C. (2018). The Determination of School Pririncipals’ and Teachers’ Attitude Towards Educational School Clubs. Unpublished Master's thesis. İstanbul Sabahattin
Zaim University, Social Sciences Institute, İstanbul.
Yaslioglu, M. M. (2017). Factor analysis and validity in social sciences: application of exploratory and confirmatory factor analyses. Istanbul University Journal of the School of Business, 46, 74-85.
Yildirim, B., & Sahin-Topalcengiz, E. (2018). STEM Pedagogical content knowledge scale (STEMPCK): A validity and reliability study. Journal of Baltic Science Education, 20(1), 38-49.
Yılmaz, H., & Çavaş, P. H. (2007). Reliability and validity study of the Students’ Motivation toward Science Learning (SMTSL) Questionnaire. Elementary education online, 6(3).
Yolagiden, C., & Bektas, O. (2021). Development of entrepreneurship scale for science lesson: a validity and reliability study. Afyon Kocatepe University Journal of Social Sciences, 23(4), 1349-1365. https://doi.org/10.32709/akusosbil.903893
Zengin, N., Kaya, G., & Pektaş, M. (2020). STEM temelli araştırmalarda kullanılan ölçme ve değerlendirme yöntemlerinin incelenmesi. Gazi University Journal of Gazi Educational Faculty (GUJGEF), 40(2), 329-355.
DOI: https://doi.org/10.17509/jsl.v5i2.39826
Refbacks
- There are currently no refbacks.
Copyright (c) 2022 Hasan Gokce, Seyide Eroglu, Melek Karaca, Oktay Bektas
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Jl. Dr. Setiabudhi 229 Bandung 40154, West Java, Indonesia