The Effect of the Augmented Reality Applications in Science Class on Students’ 21 st Century Skills and Basic Skills

This study investigates augmented reality (AR) applications on 21 st -century skills and basic skills of elementary school students. The research method is determined as a mixed method. The sample group of the research consisted of 62 randomly selected elementary school 4 th -grade students. In the experimental group, courses were taught using AR, and in the control group, the lessons were taught with activities included in the science class. In the research, the 21 st -century learning and innovation scale and basic skills scale was used as a quantitative data collection tool, and semi-structured interview and researcher's journal were used as a qualitative data collection tool. According to the results, the study of science using augmented reality applications has positively affected the skills of elementary school 4 th -grade students. The qualitative results of the research concluded that students' creativity and innovation, critical thinking, making inferences, problem-solving, collaboration, and communication skills developed. During the interviews with students and the classroom teacher, they stated that the AR application makes information more permanent by embodying the information in their minds. The lessons are fun and exciting; their interest in technology is increased and should be used in other lessons. abstract concepts, developed observation and prediction skills, and improved career awareness.


INTRODUCTION
People are asked to produce in the 21 st -century information age and question and criticize information through developing technology. Leading countries worldwide have developed different training programs on what to teach in science (Akgündüz et al., 2015). The training programs focus on STEM education and 21 stcentury skills. STEM education applies at all levels, from pre-school to higher education, including technology and engineering practices based on science and mathematics (Özsoy, 2017). In this regard, the place and types of technologies used in educational environments increase day by day and become critical. The recent use of augmented reality (AR) is also an essential aspect of technology. Augmented reality is a technology that enables real and virtual objects to interact in a simultaneous artificial environment and enables users to examine virtual * This article was produced from the master thesis of the same name accepted in 2020. objects in the real world (Azuma, 1997;Zhou, Duh, & Billing-Hurst, 2008). AR bridges the gap between real and virtual smoothly (Chang, Morreale, & Medicherla, 2010).
The concepts of augmented reality and virtual reality are expressed together but do not have the same meaning. VR is combining real-world and 3D objects in interactive virtual environments. Augmented reality is blending the real world and real-time in a computer environment and enriching it. In other words, while virtual reality transports objects to the virtual world, augmented reality focuses on enriching reality (Somyürek, 2014). AR appears to be an exciting technology for education (Luckin and Fraser, 2011). Frequent use of advanced technologies in life increases the importance of skills.
Developing and advancing technology brings innovation to societies. 21 st -century skills are the high-level skills that individuals need to acquire advanced technology, produce, select and analyze information, and use it in their daily lives (Anagün, Atalay, Kılıç, & Yaşar, 2016). 21 stcentury learning and innovation skills are defined as skills that involve individuals' life and career skills in their learning processes; and use media, knowledge, and technology skills they use in their daily lives (Atalay, 2015). Scientific process skills are the skills that individuals use to recognize problems they face, produce solutions, and make decisions (Taşdemir, 2016). The research aims to reveal the effect of augmented reality applications in science classes of elementary school students on 21 st -century skills and basic skills. For this purpose, answers to the following questions are sought. 1. Do augmented reality applications in science lessons of elementary school 4 th -grade students affect their 21 stcentury learning and innovation skills? 2. Do augmented reality applications in science lessons of elementary school 4 th -grade students affect their basic skills? 3. What are the opinions of elementary school 4 th -grade students and classroom teachers on augmented reality applications?

Research Model
In this study, embedded experimental design, one of the mixed methods, was used to examine the effect of augmented reality applications on 21 st -century skills and basic skills in science class. The embedded experimental design is a mixed method in which qualitative and quantitative data are obtained sequentially or simultaneously ( Figure 1). Data is collected to support the other group of data (Creswell & Plano Clark, 2007).

Participants
This study was conducted with 62 students in an elementary school in Antakya, Hatay, in the 2018-2019 academic year. Table 1 shows 16 female students and 15 male students in the experimental group and 12 female students and 19 male students in the control group.

Data Collection Tools
In the study, the "21 st Century Learning and Innovation Scale", which consists of 39 items and three subdimensions developed by Atalay (2015), was used to measure the effect of AR practices on students' 21 st -century skills. Meanwhile, the "Basic Skills Scale" developed by Padilla, Cronin, and Twiest (1985) and adapted into Turkish by Aydoğdu and Karakuş (2015) was used to measure the basic skills of the students. The Cronbach's Alpha coefficient of the 21 st Century Learning and Innovation Scale was calculated as 0.70. The Cronbach's Alpha coefficient of the Basic Skills Scale was calculated as 0.81 Semi-structured interview form, and the researcher's journal was also used in the study.

Implementation Process
At the first stage of the research process, the "Earth's Crust and the Movement of Our World" unit was determined from the Science Curriculum. In the second stage, lesson plans and worksheets were prepared according to the 5E model. In the third stage, the preparation process of materials for the AR application has been completed. In the fourth stage, the necessary programs for the Augmented Reality Applications, internet access, and technical support were provided to tablets and made ready for implementation. The 15-course hour lecture process in the experimental group was continued with AR applications and with the current curriculum in the control group. Hp Reveal and Space 4d + cards were used to choose the AR Application materials. The researcher prepared the experimental materials such as QR codes, fossil photographs, and mine cards. The study groups were randomly selected, and the class environment was set up to a cluster layout for students, and the implementation process continued in this way.
In the first week of the implementation process, "The ground shell of the earth's crust consists of the rock layer. It is not covered in the classification of rocks." learning outcome was processed. The QR codes prepared for the students were distributed and requested to be examined. A mind map of the word "rock" has been created. With the

Figure 1
Embedded experimental design (Creswell & Plano Clark, 2007) help of QR codes and augmented reality, the information showed that the rocks are made of minerals. During the assessment, the students completed the structured grid forms using the augmented reality application. In the second week, "Relates rocks with minerals and discusses the importance of rocks as raw materials. It is referred to many rocks and minerals in Turkey; gold, boron, marble, lignite, copper, hard coal, silver, etc., examples are given." learning outcome was processed. The students were given a video about mines, and their opinions were taken on the subject of the video. After the necessary information was provided, the material titled "Mining Cards" prepared by the researcher was presented, and the cards were introduced. Both sides of the cards are defined in the Hp Reveal program. The raw states of the mines on one side of the cards and the areas of use on the other side are shown using the augmented reality application.
In the third week, "Explains the formation of fossils. Fossil types are not covered," the learning outcome was processed. A fossil experiment was performed first in this learning outcome. After the experiment was completed, the students were given photos of the identified fossils and asked to examine them. They first predicted the species whose fossil photographs were examined, and they have also observed them through augmented reality applications.
In the fourth week, "Explains the difference between the Earth's rotation and revolution movements," the learning outcome was processed. First, the cartoon about the rotation and revolution movement of the Earth has been shown to the students in this learning outcome. Then, they were asked to give examples from daily life. The students have first introduced to Space 4d+ cards. Then, they were asked to examine the Earth and Solar System card.
The fifth week was started by repeating the difference between rotation and revolution movements of the Earth and "Explains the events that occurred as a result of the movements of the Earth." a. The rotation of the Earth is mentioned. b. The revolution of the Earth is mentioned. c. It is mentioned that the position of the Earth changes depending on the revolution around the Sun. d. The formation of day and night is mentioned. d. The concepts of day, year, and time are given." learning outcome was processed. The students were given Space 4d+ cards. Solar System card inspection requested. At the end of the implementation, one-on-one interviews were conducted with four low, four medium, and four high-level students and teachers, taking into account the students' scores on the "21 st Century Learning and Innovation Scale" (Figures  2-3).

Figure 2 Implementation process
Review of the curriculum and preparation of teaching plans according to 5E model Preparing augmented reality application materials and providing tablet support 1 st week "The ground shell of the earth's crust consists of rock layer. It is not covered in the classification of rocks." learning outcome was processed.
2 nd week "Relates rocks with minerals and discusses the importance of rocks as raw materials. It is referred to many rocks and minerals in Turkey; gold, boron, marble, lignite, copper, hard coal, silver, etc. examples are given." learning outcome was processed.

Data Analysis
As a quantitative data collection tool, 21 st Century Learning and Innovation Skills Scale and Basic Skills Scale, as qualitative data collection tool; semi-structured interview form and researcher's journal were used in the research. In addition, analyses were conducted for the research problems. SPSS software was used to analyze quantitative data, and a descriptive analysis method was used to interpret the analysis of qualitative data.

RESULT AND DISCUSSION
In this study, Shapiro Wilks Normality Analysis was performed to test whether the experimental and control group data showed normal distribution. The normality test is essential in determining which statistical formula can be used to test the research hypothesis. If the group size is less than 50, the p valu es found due to the Shapiro Wilks Normality Analysis Test, which results from normality analysis, are determined as more significant than .05. Ttests from parametric tests were used in the pre-post test scores of the experimental and control groups. The pre-test scores of the experimental and control groups are shown in Table 2.
When Table 2 is examined, there is a difference of .99 between the students' average scores in the experimental and control groups from 21 st Century Learning and Innovation Skills pre-test scores. It has been observed that the p-value based on the t-test result performed to determine whether this difference is statistically significant is .30. When the table is examined for the Basic Skills Scale, there is a difference of 2.46 points between the pre-test scores obtained by students in the experimental and control group. It has been observed that the p-value based on the t-test result performed to determine if it is statistically significant is .22. The result shows that the difference between the averages of both groups is not significant. Therefore, there is no statistically significant difference before the study regarding the 21 st Century Learning and Innovation Skills Scale and Basic Skills Scale of the students in the experimental and control groups. In the context of this skill, it can be said that the groups have similar characteristics.
According to the study, the dependent sample t-test results of the 21 st Century Learning and Innovation Skills Scale and the Basic Skills Scale pre-post test scores of the experimental and control groups students are given in Table 3.
When Table 3 is examined, there is a 5.55 point difference between the students' average scores in the experimental and control groups from the 21 st Century Learning and Innovation Skills Scale pre-post test scores. It has been observed that the p-value based on the t-test result was .00, which is used to determine whether this difference is statistically significant. There is a 2.10 point difference between the average scores obtained by the students in the experimental group from the Basic Skills Scale pre-post test scores. It has been observed that the pvalue based on the t-test result was .00, which is used to determine whether this difference is statistically significant. The result shows that the difference between pre-post test scores of the experimental group is significant. Therefore, there is a statistically significant difference between the prepost test scores of the students in the experiment group for the 21 st Century Learning and Innovation Skills and Basic Skills.  According to the study, independent sample t-test results of the 21 st Century Learning and Innovation Skills Scale and Basic Skills Scale post-test scores of the experimental and control group students are given in Table  4.

Figure 3 Application examples
When Table 4 is examined, there is a 6.32-point difference between the students' average scores in the experimental and control groups from the 21 st Century Learning and Innovation Skills Scale post-test scores. It was observed that the p-value based on the t-test result was .61, which is used to determine whether this difference is statistically significant. There is a 3.84 point difference between the students' average scores in the experimental and control groups from the Core Basic Skills Scale posttest scores. It was observed that the p-value based on the t-test result was .29, which is used to determine whether this difference is statistically significant. The result shows that the difference between the averages of both groups is not significant. Therefore, there is no statistically significant difference between the 21 st Century Learning and Innovation Skills and Basic Skills post-test scores of the experimental and control groups. Sub-dimensions of the 21 st Century Learning and Innovation Skills Scale were examined in the study.
Dependent sample t-test results of pre-post test scores regarding the 21 st Century Learning and Innovation Skills Scale of experimental group students' Creativity and Innovation, Critical Thinking and Problem Solving, Collaboration and Communication Skills sub-dimension and Basic Skills Scale Prediction and Communication Skills sub-dimension are given in Table 5.
The results of the data show that there is a significant difference between the 21 st -century learning and innovation skills scale of the experimental group students' creativity and innovation, critical thinking and problem solving, collaboration and communication skills subdimension pre-post test scores, a nd basic skills scale prediction and communication skill sub-dimension posttest scores (Figure 4).
Qualitative data has resulted in students' interest in creativity, curiosity, exposure, and technology using augmented reality in the Science class. Students expressed the effect of the application on their creativity and  Figure 5). It shows that using augmented reality applications in science class of the students has gained these skills in learning new information by doing research and noticing and eliminating misconceptions. Students expressed the positive effects of the application on critical thinking and problem-solving in semi-structured interviews as follows; S5 "Yeah, well, at first, I did not know the Earth was moving. And then I was looking at the tablet, and well, it was circling the Sun. I also learned that the Sun is constant. So, for example, I have become more interested in science…" and S6 "My friend, for instance, said that the world is from east to west, but I said it is from west to east. I knew the false information then, but then I learned the correct information." views expressed. In the researcher's journal, "They loved augmented reality application cards. Observing the movements and consequences of the Earth has helped students embody abstract terms and gain permanent information. They want to learn new things about space and the universe, and they have a discussion among themselves why Turkey does not have a spacecraft like the other countries. It was lovely and impressive." views expressed.
It was concluded that the students had improved their collaboration and communication skills in Augmented Reality Application in Science class. Students expressed the positive effects of the application on collaboration and communication skills in semi-structured interviews as follows; S7 "I was the president of that group, and I was the only one holding it, and we did it in binary order. He was very, very enthusiastic. Everybody wanted to hold it like we are taking turns, and I will hold it twice, and I will hold it three times. Yeah. I remember when my friends were looking and giving it to a friend in front of me, and he was looking, and everybody was seeing, and everybody was understanding." and S8 "Teamwork. For example, we once said who will do it, who will do it, and then we took turns doing it. Yes, it did. My communication with my friends, like, has gotten better, and I have started talking more, and I have been spending more time." views expressed. It was concluded that the students improved their prediction and communication skills in Augmented Reality Application in Science class with the qualitative data. Students expressed the positive effects of the application on prediction and communication skills in semi-structured interviews as follows: S9 "Firstly, we thought it was a baby bear, and then we looked at it as a crab. Now we see on the tablet... Many different things came out of that, which we wrote. We understood which animal it was, and we can predict better. However, it was hard because it was a regular paper. Then we looked at it from the tablet, it was very, very different." and S10 "No teacher, I had a picture of a dinosaur that looked like a flying reptile, so I wrote dinosaurs about it, I mean about my guess. I got it better. I saw something similar to marble, crab, and I wondered what it was, and I thought it was marble. So I wrote it, and I learned that it was a crab in augmented reality application. So it got better, it happened here too, when I finally looked with the augmented reality application, I found out that it was fish." expressed their views about the contributions of the implementation process. Following semi-structured interviews with the students involved in the applications and implementation process, the data were discussed under seven topics ( Figure 6).
Students stated that the implementation was realistic, provided more permanent learning to them, should be used interdisciplinary, aroused curiosity and interest, embodied S4 "We use fossils in the application. For instance, it was best to see them like that. I did not think I would do that in a science lesson. It was very beautiful. It has been great that the technology has gone this far." and S8 "I would like it to be fun with the tablet. When you talk like this, it happens with this application. I would like it with augmented reality. They are so real. That is it, on the name." expressed their views on the creativity and innovation aspects of the application.
The researcher expressed their views as "It was the first time that QR codes and augmented reality applications were introduced. They wanted to learn about what to do and what kind of a lesson it would be. The students' motivation towards the lesson increased after the application was introduced." in the researcher's journal (Researcher's Journal, 20.09.2018). The teacher stated that the augmented reality application provides prediction and curiosity, creativity, selectivity in internet usage, and increased teamwork and communication in oneon-one meetings.
Göçmen (2019), in her study titled "An augmented reality-driven design for effective learning of the solar system and beyond the subject," stated that the implementation makes learning more accessible and makes lessons enjoyable. Kurtoğlu (2019), in his study titled "The Effect of Augmented Reality Applications on Learning Processes in Information Technologies and Software Courses," stated that the implementation increases the motivation of the students, the active participation in the course, and the ambition to use the application in other courses. Omurtak (2019), in her study titled "Investigation of the effectiveness of augmented reality applications in a biology course and student opinions," stated that the implementation embodies the concepts. Demirel (2017), in her study titled "The effect of augmented reality activities supported by augmentation approach on academic achievement, critical thinking skills, motivation towards science and technology course and argumentation skills," found that academic achievement and motivation skills of students have increased. Their ability to make inferences, critical thinking skills, and argumentation skills have improved. Gün (2014), in her study titled "Effects of augmented reality applications on students' spatial abilities," stated that the implementation embodies the abstract concepts and makes the lessons interesting regarding students' views. These research results are similar to our research (Figure 7). It has been stated by classroom teachers that augmented reality applications attract students' attention. They will now use technology more consciously. They can visualize abstract concepts more efficiently in their minds, and they significantly have improved creativity. The teachers also stated that teachers should be given in-service training, and the infrastructure of the schools should be improved.
The teacher stated that the application had improved selectivity in Internet usage, creativity, and prediction skills of the students with his/her views as "The fantasy worlds are so wide. Children dream of everything. They live with dreams; some  The teacher stated that the augmented reality application contributes to the collaboration and communication skills of the students with his/ her views as "Students do the science or social studies class as group work. They are preparing their subjects. They bring their experiments. However, this app is something very different from theirs, and they liked it. They love tablets. "I looked," "You look too," "Let us look together." They had that kind of sharing. Of course, some students want to use it by themselves. This is what we encounter a lot in our daily lives. They had to share the tablets they used in the house alone but did not make an issue out of it, in turn, they somehow found a solution, and then they were again excited while comparing their predictions with the notes they took. They surprised and said, "Oh, I guessed it right." Of course, there are some problems with the children in sharing. Nevertheless, it is good that it was supported by such activity." Matcha and Rambli (2013), in their study titled "Exploratory study on collaborative interaction through the use of augmented reality in science learning," concluded that augmented reality technology has improved cooperation and communication skills in science education. These research results are similar to our research results.
The teacher stated that he/she teach fun lessons with the students through the application and that technology should be used more in education with his/her views in general as "We usually use notebooks and books when we teach the lessons. We open the Internet from the smart board where we use the most technology. Smartboard does not open everything in schools; it is restricted and does not open private channels. We do not have the opportunity to do such activities since it only opens channels allowed by the Ministry of National Education. We are not equipped in this regard either. I liked that application very much. The children saw many different things and waited with excitement. We are pleased with your work, and if we had any training on that subject, I would like to implement it in the same way. Of course, it helped. They were much happier to see them in 3D rather than seeing them from a notebook, book, or photo in one-dimensional view. It aroused their curiosity and waited with great curiosity. Parents were very happy about it. As the children were happy, they liked it too. I think it is useful. I hope they have not forgotten; they will be curious about it and continue their research. Some students will be curious and continue. It was very suitable for them." Büyükuygur (2018), in his study, stated that augmented reality application improves the vocabulary of the students by using it in foreign language teaching and makes the lessons more interesting. Sırakaya and Alsancak Sırakaya (2018) examined augmented reality use in science education on attitude and motivation and found that the students were interested in astronomy subjects. These research results are similar to our research results. Yıldırım's (2016) study showed a positive increase in students' perception and attitudes towards problem-solving skills. Squire and Jan (2007) found in their research that augmented reality technology has improved 21 st -century skills. These research results are similar to our research results. It was observed that some of the students wanted to use the technological tools and equipment used in the application individually, and they could not be problem-solving groups much. However, it was determined that students who wanted to work individually in other weeks were included in the group, and groups have solved problems they encountered by collaborating. Another result obtained under the title of cooperation and communication is that because students are in randomly created groups, their communication with their classmates increases, and a positive classroom climate is achieved.
Ibanez and Kloss (2018), in their research titled "Augmented reality for STEM learning: A systematic review," compiled research on the use of STEM and augmented reality and suggested the use of augmented reality application in the STEM discipline. Linder, Rineow, and Jürgens (2019) were highlighted the relationship between augmented reality and STEM activities in their research. These research results are similar to our research results. Delello (2014), in her study titled "Insights from pre-service teachers using science-based augmented reality," concluded that the implementation positively affected the classroom environment and increased motivation and participation. Demirer and Erbaş (2015), in their research titled "Investigation of Mobile Augmented Reality Applications and Evaluation of Educational Perspective," recommended applications that should be used in an educational environment. Kara (2018), in his research titled "Investigation of research on the use of augmented reality practices in education," concluded that the effect of augmented reality applications is a matter of curiosity in education, and the studies have increased since 2012. The studies have been focused on science and engineering. Soylu (2019), in his research titled "The effect of the education program based on augmented reality practices on pre-school teacher candidates' attitudes and views," concluded that pre-school teacher candidates' interest in technology increased. These research results are similar to the results of the study. Based on the findings and results obtained in the study, the following recommendations can be made.

CONCLUSION
The research investigates the effect of AR on students' 21 st -century learning and innovation skills and basic skills. After implementation, the experimental group students' 21 st -century learning and innovation skills differ significantly between pre-test and post-test scores. It reveals that the effective use of augmented reality in lessons has positively affected 21 st -century skills. There was a significant difference in favor of the post-test score was observed when pre-test and post-test scores of the experimental group were compared regarding the results of quantitative data on the dimensions of "creativity and innovation", "critical thinking and problem solving", and "collaboration and communication", which are identified as the sub-dimensions of the 21 st -century learning and innovation skills scale It was noticed in the activities that the students realized their misconceptions at the end of the implementation process, began to question the accuracy of the information they encountered and gained critical thinking skills. It was also observed that the students learned when they were curious about it throughout the process from various sources through questioning and research. It was concluded that while students had difficulty solving the problems they encountered in the activities and during the implementation process, they tried to solve and reached a solution by finding different solutions towards the end of the process.
When the basic skill scores of the experimental and control group students were analyzed within the groups, there was no significant difference in the basic skills scale scores of the students in the control group. In contrast, a significant difference was found in the scores of the experimental group students. This result shows that augmented reality application has a positive effect on students' basic skills.

ACKNOWLEDGEMENT
The present study was not supported by any foundations, institutions, or others. Therefore, there is no funding to report for this submission. We confirm that all co-authors contributed equally to the study. To the best of our knowledge, there is no conflict of interest..