Online Flipped Classroom: Developing Postgraduate Science Education Students’ Critical Thinking Skills

During the Covid-19 pandemic, online learning should be carried out through innovative methods. This study aims to determine whether implementing an online flipped classroom, with converting all face-to-face sessions into virtual face-to-face sessions, can facilitate the development of critical thinking skills of postgraduate science education students in the School Physics course. This quantitative study used one group pretest-postest research design. The subjects were postgraduate science education students who took School Physics courses in the first semester at one of the public universities in Indonesia. Online Flipped classroom was supported by Google Classroom Learning Management System (LMS). Critical thinking skills data in School Physics were obtained through tests. Data were analyzed using a description of normalized changes (c) and nonparametric-inferential analysis with the Related-Samples Wilcoxon Signed Rank Test. The results showed that the online flipped classroom could facilitate the develop ment of the students’ critical thinking skills with an average normalized change of .71 in a high category. Even though the students felt the benefits of an online flipped classroom, they still wanted a face-to-face learning mode to be carried out soon.


INTRODUCTION
Higher education is responsible for developing students' critical thinking skills (Bassham, Irwin, Nardone, & Wallace, 2013).Necessary thinking skills are essential in coping with academic or professional interests (Costa & Kallick, 2014;Kraisuth & Panjakajornsak, 2017) and become an indicator of lifelong learning (OECD, 2018).Postgraduate students majoring in Science Education are prepared to become academics who can later act as researchers, lecturers, educational consultants, or teacher advisors.As shown in the overview of The University of British Columbia's Master of Education in Science Education (MEd), the postgraduate interest is a wide range of research and professional interests in the STEM education field (UBC, 2020).In addition, Moeti, Mgawi & Waitshega (2017) and Ashour & Hawamdeh (2012) emphasize that postgraduates need to perceive critical thinking skills to be professional researchers or lecturers.The learning process conducted in Science Education major at the Universitas Negeri Surabaya postgraduate program focuses on developing critical thinking, creative thinking and problem solving, communication, and collaboration skills (S2pendsains, 2020).
Physics is a way to understand our world and recognize the underlying principles and laws that connect to the disparate phenomena of our physical world, which is full of content and meaning (Rau, 2017).It is an authentic, essential, logical, and rational science that underlies the development of science and technology.In connection with its materials, problem-solving and decision-making skills can be carried out through some stages of a thinking process, which directly promote critical thinking skills and conceptual understanding of physics (Magno, 2010).Critical thinking requires students to be thorough, purposeful, and deliberate, focus on the main issue, and evaluate all parts of its complex and challenging claims and arguments (Sasson, Yehuda, & Malkinson, 2018).In a scientific context, the evaluation aspect involves interpreting data, drawing accurate conclusions from the data, comparing and evaluating models and data, estimating methods, and deciding how to proceed in an investigation (Walsh, Quinn, Wieman, & Holmes, 2019).In a science classroom, students' critical thinking can be identified by their ability to respond to daily life problems (Suhirman, Prayogi, & Asy'ari, 2021).
Regardless of the education level, the learning process is currently required to adapt to the conditions of the Covid-19 pandemic.The Covid-19 outbreak was first detected in Wuhan City, China, in December 2019 and was declared a pandemic by WHO on March 11, 2020.In education, various countries have responded to adjusting the learning process according to the conditions of their respective countries.In Indonesia, the learning process is conducted from home, based on the Letter of the Minister of Education and Culture of the Republic of Indonesia Number 4 of 2020 concerning the Implementation of Education Policies in an Emergency for the Spread of Coronavirus Disease (Covid-19).In higher education, the home learning process is carried out online.Online lectures have been practiced for a long time, but during the Covid-19 pandemic, this method has been compulsory in many countries.Kuntarto (2017) states that online learning is more effective and able to increase students' mastery of subject material compared to the use of a face-to-face learning mode.The online lecture system positively contributes to encouraging disparities in the quality of higher education in Indonesia (Mustofa, Chodzirin, & Sayekti, 2019).Rosdiana, Widodo, Nurita & Fauziah (2018) state that learning vibrations and waves through online learning improves science teacher prospectives' problem-solving and graphing skills.However, various obstacles arise when a learning process is carried out entirely online without a face-to-face presence for a long time.The most prominent obstacle is that learning activities are too boring for students.Livana, Mubin, & Basthomi (2020) state that 55.8% of students complained that online learning was boring.Cao et al. (2020) also convey that academic activity delays were positively related to student anxiety during the Covid-19 pandemic.The cause of students' stress might involve academic problems, interpersonal problems, learning activities, social relationships, encouragement and desire, and group activity (Yusoff & Rahim, 2010).
On the other hand, it was also found that, in general, assignments did not have a significant effect on student stress levels (Mardiati, Hidayatullah, & Aminoto, 2018).Therefore, the idea of learning to bridge the boredom of online lectures, appropriate assignments, and adequate results is necessary when the situation demands a complete online learning process.Therefore, one of the ideas applied in this study is an online flipped classroom.
The flipped classroom is a learning approach that uses an online platform with flexible time (Karabulut-Ilgu, Cherrez, & Jahren, 2018).In its application, students complete several online activities to prepare the face-toface learning on campus.It is dissimilar to the traditional lecture, where a professor must present new materials (Reidsema, Kavanagh, Hadgraft, & Smith (Eds), 2017).Instead, in a flipped classroom, instructors provide online materials for students to read, watch, outline, summarize, understand, and present on their own so that the classical method can be allocated to student-centered learning activities (Baytiyeh, 2017).Although studies on flipped classrooms are still required, most publications (e.g., peerreviewed) describe it as a type of blended learning (Abeysekera & Dawson 2015).
The application of flipped classrooms in lectures has several advantages, for example, offering flexibility, increasing interaction between lecturers and students as well as students and students, and developing professional skills (Karabulut-Ilgu, Cherrez, & Jahren, 2018).Baepler, Walker, & Driessen (2014) found that students who learned using flipped classroom model had better performance than those who knew with the traditional model.The application of flipped classrooms is considered appropriate because students can study literature or lecture assignments efficiently and flexibly.They also can discuss the materials or tasks face-to-face.Marlowe (2012) shows that flipped classrooms could reduce students' stress levels and increase their semester scores.However, the nature of flipped classrooms cannot be applied in many countries due to the existing Covid-19 pandemic.
In the flipped classroom, there is a face-to-face session.However, during the Covid-19 pandemic, many countries still do not allow face-to-face meetings.Therefore, the face-to-face session in flipped classrooms is replaced by face-to-face online sessions called Online Flipped Classroom.For example, Reidsema, Kavanagh, Hadgraft, & Smith (Eds), ( 2017) have not included face-to-face online ideas in the flipped classroom section in books on flipped classrooms for higher education.To the researchers' knowledge, there is little research on an online flipped classroom.Weinhandl, Lavicza, & Houghton (2020) conducted a study about an online flipped classroom for mathematics class, which is different from the present study's focus.Smith (2020) also promoted online, face-to-face sessions that showed effective learning times and better student performance.Therefore, implementing a fully online flipped classroom with a duration of one semester in physics lectures to facilitate the development of critical thinking skills is worth researching.Thus, this study aims to determine whether the implementation of the online flipped classroom in School Physics lectures can develop science education postgraduate students' critical thinking skills.

METHOD
The study was pre-experimental, using one group pretest-posttest design.This design in Table 1 let the participants be measured on their outcomes before and after the treatment (Mertens, 2010).
The research subjects comprised 13 students who took the School Physics course for the academic year of 2020/2021 at the Science Education Postgraduate Program at one public university in East Java Province, Indonesia.
Regarding gender, the ratio of male and female students was 5:8.The treatment in this study was using an online flipped classroom with School Physics materials.The treatment encompassed several learning strategies: learning the material, conducting investigations, creating presentation files, carrying out an online face-to-face discussion, and accomplishing assignments.The response variable of the present study was critical thinking skills in School Physics, defined as the student's skills in explaining, making arguments, and assessing the truth shown by the critical thinking skills test score in School Physics.The researcher hypothesized that the treatment was effective, so the outcome scores improved.On the contrary, the treatment had no effect when the scores did not change.However, the researchers did not claim that the online flipped classroom was the best strategy among other strategies for online learning.
The online flipped classroom strategy was implemented in the lecture process by changing lectures from face-toface to online, according to the flow chart in Figure 1.In the online flipped classroom research, learning began with introductory sessions carried out synchronously.Based on the introduction, students conducted inquiries and searched for learning resources to make presentation files on certain topics according to the results of the distributed tasks.The lecturer gave feedback on the uploaded presentation files, and then the student corrected them according to the lecturer's suggestion.Afterward, face-to-face online learning was conducted where students presented their work while the other students responded.Lecturers provided reinforcement, clarification, and further suggestions if necessary.This online face-to-face was continued with the task of analyzing, making explanations, and initiating arguments related to potential misconceptions according to physics topics discussed asynchronously.These activities were carried out using Google Classroom Learning Management System (LMS).This LMS was chosen because it was easy, quickly accessible, and integrated with various hardware owned by students.
Students conducted studies encompassing concepts, principles/laws along with their applications, and potential misconceptions on the topics, including 1) kinematics, 2) dynamics, 3) heat and thermodynamics, 4) waves, sound, and optics, 5) electricity, 6) magnetism and electromagnetic induction, and 7) earth and space.The inquiry activity in this study was an inquiry designed independently by students according to the topic using existing equipment and materials according to the context and virtual laboratories.Students were required to formulate problems, hypotheses, and methods to test hypotheses, present data, analyze data and draw conclusions.Thus, students were expected to develop their abilities in building arguments based on deductive and inductive reasoning.A rubric assessed this activity that two experts on science education had validated.
The activities of lecturers and students were observed based on the activities at LMS by detailing the personalization o f lecturers and students in the pre-class and in-class phases based on Koh's (2019) idea, however, focusing on various activities.Critical thinking skills were measured through a critical thinking skills test in School Physics.The test instrument used consisted of 11 questions on critical thinking skills, of which the indicators of essential skills of thinking referred to Ennis (2011), namely: 1) making arguments equipped with mathematical models based on deductive analysis of physical symptoms; 2) making arguments equipped with a mathematical model based on an inductive analysis of physical symptoms; 3) explaining the statement of principles/laws of physics, and 4) assessing the truth of the statement accompanied by arguments based on the concepts/principles/laws of physics.The test questions consisted of a series of question formulation according to the indicators, internal reviews, revisions, and expert validation and revision.Pre-test questions differed from post-test ones; however, both are equivalent.In addition, science education experts validated the content, construction, and language use questions.The further assignments: conducting analysis, making explanations, and initiating arguments related to potential misconceptions according to the topics discussed (asynchronous) This study also used a questionnaire to reveal students' responses to the School Physics learning process with an online flipped classroom strategy.This response was mandatory to obtain an overview of what science education graduate students experienced and felt regarding the learning process using the online flipped classroom strategy.In addition, the response was essential to get feedback that could be improved in the future.The questionnaires were administered online through the LMS.
This study used descriptive and inferential data analysis techniques.Descriptive analysis was a description of increasing critical thinking skills using the normalized change (c) equation proposed by Marx & Cummings (2007) (see table 2).Sriyansyah & Azhari (2017) also suggested normalized change analysis compared to normalized gains.
The inferential analysis was a median difference between the post-test and pre-test scores using nonparametric-inferential analysis of the Related-Samples Wilcoxon Signed Rank Test (Siebert & Siebert, 2018).This   Because of the small data, the researcher could not assume that the subjects were normally distributed and homogeneous, so a nonparametric analysis was undertaken.The significance level was .05.The inferential analysis was carried out using SPSS version 26.0.

RESULT AND DISCUSSION
Table 3 shows the online flipped classroom implemented in the first semester of 2020/2021.Table 3 depicts that the online flipped classroom took place with students who conducted inquiries and studies first, followed by virtual face-to-face presentations and discussions.Then, the activity was continued with a followup assignment.This pattern followed the Flipped Classroom strategy according to Bergmann & Sams (2012), Reidsema, Kavanagh, Hadgraft, & Smith (Eds), ( 2017), and Baytiyeh ( 2017), but every learning stage was conducted online, named an online flipped classroom.Figure 2 shows an example of the online flipped classroom along with its example of the results of students' collective assignments in LMS.
Critical thinking aspects related to providing explanations, making deductive arguments, and making arguments inductively were developed through online inquiry activities in the form of investigating gas simulations in closed spaces.Table 4 portrays the results of the student inquiry report data analysis.
Table 4 conveys that, on average, students obtained online inquiry results of 97 with a standard deviation of 4.9, which were in a very good category.The point that was emphasized during the online discussion was analyzing data.Some students took logical leaps in data analysis by deductively mentioning specific laws.Supposedly, in the data analysis phase, students formulated a mathematical model based on data trends (e.g., graphs).That was also to develop the ability to make arguments based on students' inductive reasoning.
Table 5 shows the results of the descriptive analysis of the pre-test, post-test, and c values for students' critical thinking skills.5, it was found that there was an increase in students' critical thinking skills, which was indicated by the elevating average pre-test score of 42 to 83 in the posttest.Such improvement was categorized in a high category, with the average normalized change (c) of 0.71.
A difference test in pre-test and post-test scores was conducted to determine whether online flipped classroom learning significantly affected students' critical thinking skills (See Table 6).In addition, hypothesis testing of the median difference between pre-test and postest scores was carried out using Related-Samples Wilcoxon Signed Rank Test.The hypotheses were drawn as follows: Ho: there is no median difference between pre-test and post-test scores.
Ha: there is a median difference between pre-test and post-test scores.
Table 6 shows that Ho was rejected, and Ha was accepted.Thus, the post-test and pre-test scores for critical thinking were significantly different.These results reinforced the findings of He et al. ( 2017), who claim that flipped classrooms in architectural physics produced better students' learning performances and abilities than the traditional classes.This study showed deeper learning in conducting investigations, browsing the literature, formulating presentation files, and discussing synchronously in the virtual face-to-face phase.The strength of the online flipped classroom to facilitate students to search various literature and conduct collaborative online investigations was also depicted by Sun, Wu & Lee (2017).Sun, Wu & Lee. (2017) argued that the flipped classroom model promoted a learning environment that prompted its learners to seek external In addition, the lecturers believed that the constructive feedbacks were helpful for the students."In my opinion, synchronous face-to-face lectures let me understand the material discussed easily because, when the discussion is over, the lecturer still provides the explanation and restrengthens any misconceptions.That is attractive to me".That means that this online flipped classroom provided opportunities for lecturers to provide feedback to reinforce students' motivation (Tricomi & DePasque, 2016).Zainuddin, Haruna, Li, Zhang, & Chu (2019) emphasized that the flipped classroom yielded positive learning outcomes in students' learning activities, such as learning motivation and engagement, social interaction, and self-directed learning skills.
Figure 3 shows Radar Score diagram based on critical thinking indicators Indicator 1: Making arguments supported by a mathematical model based on deductive analysis of the physical context; Indicator 2: Making arguments supported by a mathematical model based on an inductive analysis of the physical context; Indicator 3: Explaining the statement based on the principles/laws of science-physics; Indicator 4: Assessing the validity of statements that were strengthened by arguments Following Figure 3, each average score on indicators of critical thinking portrayed an increase.Normalized change (c) to arguments supported with a mathematical model based on a deductive and inductive analysis of physical phenomena was in a high category.That indicated that providing opportunities for students to carry out activities to formulate problems, generate hypotheses, conduct investigations to obtain data, analyze data, and conclude inquiry activities had an impact on the ability to build arguments accompanied by inductive mathematical models.This fact shows that implementing online learning with flipped classrooms could give students scientific investigation skills.These results were in line with a study by Tan, Yangco, & Que (2020), which revealed that online science learning with flipped classrooms could improve students' science process skills and conceptual understanding.Marshall & Kotska (2020) also said that online learning with flipped classrooms strengthened teachers' presence virtually in a class.In other words, when students worked on assignments monitored by lecturers online at LMS, students felt the presence of lecturers.Many relevant studies stated that deductive thinking skills required continuous practice (Eglington & Kang, 2018;Carriera, Amando, & Jacinto, 2020).This study portrayed that tracing learning resources, making presentation files, and discussing them facilitated the students a lot in training their critical thinking.Moreover, McLean, Attardi, Faden, & Goldszmidt (2016) showed that flipped classrooms produced better learning processes and academic performance (Romero-García, Buzón-García, & Touron, 2018).
Although the online flipped classroom facilitated students' learning activities, lecturers' feedback, and increased critical thinking skills, students still preferred offline face-to-face meetings.Some students said, "... it is better face-to-face; eye fatigue; unstable network constraints; extravagant; although flexible, the combination with face to face is better; can't feel interpersonal relationship... ".Therefore, in the context of online learning, it was still in line with Susanti, Hidayati, Anggreiny & Maputra, (2020), who claimed that 83.6% of students stated online learning in the last semester was not effective because of difficulty in understanding learning materials, neglected task, boring, and no teacher's explanation.Troublesome signals, not optimal face-to-face learning, overload task, low economy, challenging to ask questions, and less attention to grades.Irawan, Dwisona & Lestari (2020) showed that students started to get bored during online learning after the first two weeks of the learning process.Although the online flipped classroom implementation showed that students wanted some offline, face-to-face activities (to feel good, not to get tired, and not to have the feeling of interpersonal relationships), they showed good learning performance.They produced better critical thinking skills in School Physics.

Figure 1
Figure 1 The implementation of the online flipped classroom used in the present study

Figure 2
Figure 2 Examples of online flipped classroom in LMS

Figure 3
Figure 3 Radar Score diagram based on critical thinking indicators

Table 1
Experiment design O1 X O2 Initial measurement (pretest) of critical thinking skills related to School Physics The treatment given using an online flipped classroom in School Physics course The final measurement (posttest) of critical thinking skills related to School Physics

Table 2
Equations and criteria for normalized change

Table 3
Implementation of online filpped classroom

Table 4
Inquiry results That was consistent with what one student said, "...I think it is exciting.Honestly, this is my first time with a system like this implemented.The lecture was unique and challenging, making me have to search various sources, question ideas in my mind, and discuss with friends… ".From this statement, the online flipped classroom allowed students to self-regulate, foster selfefficacy, and increase their learning outcomes, of which Chen & Hwang's (2019) study also revealed similar findings.