Exploring student achievement and perceptions in an online flipped grammar course

Flipped learning to date has generally been implemented within a blended classroom. However, the growing number of fully online courses and the existence of Covid-19 have shown the need to try using flipped learning in a fully online class. This paper aimed to explore student achievement and perceptions in an online flipped grammar class. A paired-sample t-test showed a significant difference between students’ pre-test and post-test scores with a medium effect size after attending the online flipped course for one month. The interview and questionnaire data revealed students’ overall positive perceptions toward the introduction of flipped learning within the course. Students mentioned that the flipped approach made the course more interactive and provided them opportunities to ask questions to the teacher with an immediate answer. They also expressed their positive attitude towards the use of different learning platforms (e.g., LMS, Zoom) and modes (synchronous and asynchronous) that could ease their learning experience within the course. Nevertheless, students recommended that the virtual meeting via Zoom needs to be limited in time duration.


INTRODUCTION
Considering technology literacy as a twenty-firstcentury need, researchers and instructors have been searching for innovative approaches to the integration of technology tools that could meet both teachers` and students` demands for more effective and efficient teaching and learning practices. The appearance and continuous growth of fully online learning such as Massive Open Online Courses (MOOCs) have further shown the need to meet this demand. One of the emerging approaches in the field of educational technology and language teaching both in blended and fully online learning contexts is flipped learning (Cheng et al., 2020;Kostka & Marshall, 2018).
In flipped learning for blended contexts, students are required to acquire knowledge at home usually by watching pre-recorded video lectures made by the instructor prior to attending a face-toface class. In this way, students can flexibly learn at their own pace in that they can watch and replay the videos at will to sharpen their understanding. In class, teachers can then focus more on helping students engage in a practice, group discussion, and collaboration . In the field of language teaching and educational technology, learning through practice, discussion, and collaboration has been considered to be able to promote language learning (Laurillard, 2012;Lightbown & Spada, 2013), thus showing the potential of this approach.
Unlike flipped learning within blended contexts (henceforth, traditional flipped learning), in fully online contexts (henceforth, online flipped learning) students and teachers do not meet face-toface in class but meet through an online web conferencing application (e.g., Zoom) (Stöhr et al., 2020). In online flipped learning, students will be able to access pre-recorded videos through a learning management system (LMS) and complete automatically graded online quizzes. In the video web conferencing, the teacher and students discuss the quizzes that students have completed in advance. This shows that online flipped learning offers both synchronous and asynchronous online learning.
Despite the growing number of studies that have explored flipped learning (e.g., Hsieh et al., 2017;Wang et al., 2018;Yang et al., 2018), most of these studies were conducted in a traditional flipped context rather than in an online flipped environment. Some research (e.g., Stöhr et al. 2020) has also started to focus upon the implementation of the flipped approach in online flipped learning classes, yet there have not been many published studies on this topic, particularly within the field of language teaching. Currently, both the growing number of MOOCS and the backdrop of the COVID-19 pandemic have indicated the need to attempt to implement flipped learning in a fully online course.
Given the dearth of studies within the field and the potential of flipped learning as an innovative teaching approach, the current study aimed to implement this flipped learning approach to teaching grammar in an online flipped course. Specifically, this study explored both student learning achievement and perceptions of the implementation of this innovative flipped learning through a semi-structured interview and questionnaire. Thus, it utilised a mixed-method approach. The findings are expected to become practical evidence for teachers and curriculum makers to make informed pedagogical decisions to adjust teaching strategies and materials that suit both teacher and student needs for online flipped learning.

Flipped learning
Flipped learning has started to gain a reputation in tertiary education contexts as it has been seen as an alternative to conventional teaching methods. Traditionally, students acquire knowledge by listening to the lecture in class and practice it at home by having assigned homework (Garrison & Vaughan, 2008). In flipped learning, students acquire knowledge at home prior to attending their classes. Although students may arguably learn through a number of media such as PDFs, web pages, textbooks, audio, and others, flipped learning usually implements videos with typically short durations. Offering videos as the medium for students to acquire knowledge at home, flipped learning has been considered to be able to engage and motivate students, for millennials in general allocate most of their time playing with their mobile phones or laptops (Hung, 2015).

Effectiveness of traditional flipped learning
One of the studies that has investigated the effectiveness of flipped learning in blended language learning contexts was by Hsieh et al. (2017). Hsieh et al. compared flipped and conventional instruction contexts and found that the group with flipped learning had significantly higher scores in their post-test compared to the group with the conventional instruction. The authors, however, noted that both groups did show significant improvement in the post-tests. Hsieh et al. specifically investigated idiomatic proficiency. Similarly, in oral proficiency Wang et al. (2018) also reported that students instructed with a flipped learning approach significantly outperformed students with traditional teaching. Wang et al. showed that the flipped group had the advantage for fluency and coherence, lexical resources, and grammatical range and accuracy, while pronunciation was not found to be statistically significant.
Comparing traditional and flipped groups, Yang et al. (2018) also found that the flipped group performed significantly higher in the post-test for speaking, writing, and reading; no statistically significant differences were found for listening skill. Related to this improvement, the authors stated that students with flipped learning were more engaged with learning materials and better prepared prior to attending their class. A similar finding was also reported by Hung (2017) who focused upon the investigation of how flipped learning could help improve university students` English spoken ability in Taiwanese contexts. Hung showed that the group with flipped learning gained significantly higher speaking scores in their post-test compared to the group with a traditional teaching model. The author also stated that the flipped group showed a higher willingness to communicate within a class than those in the conventional group.
Thus, the above findings have shown how flipped learning in blended contexts could enhance student achievement in a language programme. However, none of the above studies was conducted in a fully online class. This shows the gap and the need to conduct a study investigating the implementation of the flipped approach in a solely online course.
Collaboration, practice, discussion, engagement, and autonomy Laurillard (2012) argues that teaching must be considered `a design science`. This means that any pedagogical decisions in the selection of teaching strategies and materials should be based upon tested theories. In other words, any decisions in teaching practices must be research-based. This is Copyright © 2021, authors, e-ISSN: 2502-6747, p-ISSN: 2301-9468 particularly crucial when instructors decide to integrate technology platforms into their teaching practices.
In the field of language teaching, teachers must act based upon second language acquisition (SLA) theories. SLA theories show that active learning approaches such as collaboration, practice, engagement, discussion, and student learning autonomy have been considered to be able to promote effective language learning (Lightbown & Spada, 2013).
Because flipped learning has also been reported to be able to facilitate active learning (Hung, 2015), the current study then aims to investigate how flipped learning could actively engage students with the course materials. Related to this objective, a growing number of studies have employed interviews and survey questionnaires to investigate students' perceptions regarding the implementation of flipped learning.
For example, Wang et al. (2018) reported that the flipped group spent more time out of the classroom preparing for their class. This shows that flipped learning could help engage students in their learning processes and become more autonomous. Hung (2017) also reported that students in flipped learning environments had better motivation to collaborate with their peers in practising their spoken skill in class. This could be because they had already had preparation outside class. Hung also noted that this group showed statistically significantly higher satisfaction scores in survey responses. Thus, both of these studies have indicated how students are more willing to engage with a flipped learning approach.
Employing quantitative and qualitative surveys, Webb and Doman (2019) found that students were satisfied with the flipped approach, and they even also said that they wanted more flipped contents. Students also expressed that they were aided by the use of technology in their mastery of contents. Similarly, Hung (2015) showed that students were more autonomous in their learning processes as they reported that they spent approximately 68 minutes outside class learning flipped videos prior to coming to their class. These students also commented that they spent more time on classes that utilised flipped materials than for other classes with traditional teaching methods. These students said that they were more confident to collaborate and have a discussion in their class as they had watched the videos before class.
The above studies have shown students` satisfaction, better engagement, autonomy and willingness to collaborate and practice in their classes. Despite these potential benefits, there might also be challenges that need addressing prior to the implementation of flipped learning. Dooly and Sadler (2019) found that teachers in their study at times could be ill-prepared when implementing the approach as they had to provide videos a few days before class.
Thus far, a large number of studies have investigated flipped learning. Nonetheless, most of them have not dealt with how the flipped approach could be conducted in a fully online course. They have mostly focused upon blended learning contexts, or traditional flipped learning environment. Hence, it shows the need to conduct research investigating the implementation of flipped learning in a fully online course.

Online flipped learning
Because of the need to teach a fully online class, now more and more researchers have attempted to think how flipped learning approach could also be implemented within a solely online course. To date, this approach has generally been utilised within a blended course. Marshall (2017) and Marshall and Rodriguez-Buitrago (2017) developed a framework for online flipped learning that they named Synchronous Online Flipped Learning Approach (SOFLA). Basically, students learn or watch video lessons at home on a learning management system (LMS) and complete some related quizzes. Students and the teacher then discuss the quizzes via a virtual classroom, or a web video conferencing. Thus, this framework shows that the online flipped approach combines synchronous and asynchronous learning formats.
Marshall and Rodriguez-Buitrago (2017) also reported that the students had overall positive perceptions of the course. The students felt that they enjoyed interacting with their teacher as well as with other students. The authors also found that the students liked the class because they could learn and access the materials from different formats such as video lessons, slides, and websites (LMS). They also expressed that with the virtual classroom they could ask questions or clarify directly.
Comparing student satisfaction between traditional and online flipped learning, Swart and MacLeod (2020) concluded that there was no significant difference in student satisfaction between traditional and online flipped learning. They also showed that flipped teaching could lead to improved learning outcomes. In their conclusion, Swart and MacLeod mentioned that "our review of the literature found no research on this, perhaps due to the limited attempts at flipping online courses" (p. 132). Thus, it shows the dearth of studies investigating online flipped learning.
Another study by Stöhr et al. (2020) evaluated the efficacy of the online flipped classroom as compared to the campus-based flipped format (traditional flipped learning). The authors found that both formats (campus-based and online flipped) could significantly improve student performance, showing the potential of the online flipped learning as an innovative approach that merits further Copyright © 2021, authors, e-ISSN: 2502-6747, p-ISSN: 2301-9468 investigation. Regarding the lack of research within the area, they stated that "we are not aware of any previous study attempting to evaluate the efficacy of the online flipped classroom in a more rigorous way" (p. 8).
Unfortunately, we did not find many published empirical studies that had also investigated the implementation of online flipped learning, albeit we had tried to check the articles that cited Marshall's work. We also tried the keyword "synchronous online flipped" on Google Scholar, yet no studies were of our interest. Lastly, we searched for the keyword "online flipped learning", and we found only two (Swart & MacLeod, 2020;Stöhr et al., 2020) that appeared similar to Marshal`s studies that attempted to implement the flipped learning approach in a fully online course.
Thus far, the implementation flipped learning in a fully online course remains understudied. It became the reason why the current study implemented flipped learning in a fully online grammar course. We aimed to investigate whether students could benefit from this approach, and we also explored students' perceptions of this implementation. Specifically, this study investigated the following three research questions: 1. How does the implementation of online flipped learning improve the students' grammar achievement? 2. What are the students' overall perceptions of the course? 3. What are the students' perceptions of the online flipped course?

METHOD Research design and instruments
To answer the first research question "how does the implementation of online flipped learning improve students' grammar achievement?", we employed a pre-experimental design with a one-group pretestposttest design. Thus, our study utilised tests as the instruments to answer the first research question.
As an exploratory study, the finding of our research is expected to show whether the introduction of flipped learning in a solely online course merits further investigation (Phakiti, 2014). With regard to the test validity and reliability, both of the tests were official tests published by ETS (Education Testing Service) (ETS, 2010); hence, the validity could then be warranted.
For the second and third research questions related to students' perceptions, we utilised both semi-structured interviews and open-ended questionnaires to gain learners' perceptions. We allowed students to opt for either an interview or a questionnaire. That is, our participants could tell us their perceptions of the introduction of flipped learning by either attending an interview or filling in the questionnaire. (The list of interview and questionnaire questions can be seen in Appendix A & B).
The teacher in this course is the second author of the current study; thus, this research can be considered an autoethnography study. Coia and Taylor (2009) mentioned that autoethnography is a research method that involves self-observation and reflexive investigation to improve or at least better understand the teachers` own practice. This research can also be regarded as a mixed-method study, for it utilised both quantitative and qualitative data to answer the research questions. The use of more than one method or resource (e.g., interviews, questionnaires) for data collection (triangulation), could help us gain insight from more samples, thus increasing the credibility of our research findings (Creswell, 2012).

Site and participants of the study
This study was conducted in a solely online grammar class (structure and written expression) as part of the TOEFL ITP (Test of English as a Foreign Language -Institutional Testing Program) programme at Edinmelb course (kelasdaring.edinmelb.com). Edinmelb, a Moodlebased course, is a private, fully online course that is open to Indonesian learners. (The appearance of the website can be seen in Appendix C).
Regarding the research participants, we received 531 applicants from different areas throughout Indonesia with different professions (e.g., teachers, university students, doctors, civil servants, fresh graduates) who were interested to enrol in the TOEFL class and were willing to participate in this research. The registration of the programme was open for three days, from the third to (3 rd ) to the ninth (6 th ) of September 2020. All of the applicants were required to take a pre-test with 40 items, or questions, for twenty-five (25) minutes for structure and written expression.
To be considered eligible to join this TOEFL programme, the applicants needed to have at least thirteen (13) correct answers in their pre-test. This minimum score was set to ensure that the applicants were ready for the TOEFL class, which requires learners to have a certain knowledge of English (Tannenbaum & Baron, 2011). Of the 531 applicants, 183 were eligible and invited to join the class and participate as participants within the current research. The participant age ranged from eighteen (18) to forty-one (41) years with an average of 23.38. For ethical concern, students who were under eighteen years of age were excluded from the research data, albeit they were still accepted to join the course.
Of the 183 students who were enrolled in the class, sixty-five (49 females and 16 males) who completed the post-test; hence, the current study had 65 participants for the first research questions. For the second and third research questions, we received Copyright © 2021, authors, e-ISSN: 2502-6747, p-ISSN: 2301-9468 twelve students who were willing to volunteer for the interviews, yet because of the schedule issue, we only managed to interview seven of them. For the questionnaire, we collected fifty-seven responses. Thus, in total there were 64 participants for the second and third research questions.

Instructional techniques
The teaching materials were in the form of screenrecording videos that contain the explanation of the grammar topics by the teacher (henceforth, video lessons). Each of the grammar points was taught deductively and explicitly in the Indonesian language, the students` mother tongue. Each of the videos was followed by an untimed 10-question quiz that students were required to complete. All of the quizzes were accompanied by a video that provided the explanation to the correct answers of the quizzes (henceforth, video quizzes).
Before the implementation of the newly introduced approach, flipped learning, this class used to be fully asynchronous, that is, learners accessed all the materials through the learning management system without having any synchronous meeting with the teacher or other students. The interactions were mostly established through a discussion board on the website and sometimes via WhatsApp chat. With the introduction of the online flipped learning approach, the course started to offer a blend of the two instructional techniques (synchronous and asynchronous).
What made this class be considered `a flipped classroom` is that the synchronous meetings were not intended to cover all the explanations of the grammar points, yet the synchronous meetings through video conferencing (Zoom) were offered only to cover the explanations of the quizzes, not the video lessons. The video conferencing was conducted three days a week for approximately 60 to 90 minutes per meeting. It covered several quizzes for some grammatical topics that students had completed in advance.
Similar to that of the traditional flipped classroom in which students watch pre-recorded videos at home and then have practice and discussion in the face-to-face class (Garrison & Vaughan, 2008), in the online flipped classroom students watch the video lessons at home and complete the quizzes. The discussion of the quizzes was conducted through a virtual classroom (Zoom). This online flipped classroom has been called Synchronous Online Flipped Learning Approach (SOFLA) (Marshall & Kostka, 2020).

Instructional materials and treatment
The selection of the taught grammar points along with the quizzes was mostly adapted from the TOEFL preparation book written by Phillips (2004). Prior to the first Zoom meeting on 12 th September, students learnt five grammatical topics (or five video lessons) and completed five different quizzes for each topic at home through the website, or LMS from 8 th September. After completing the quizzes, students could access the video quizzes that contained the explanations to the correct answers to the quizzes. These first five topics were all about simple sentences, divided into five small topics: subjects and verbs, prepositional phrases, appositives, present participles, and past participles.
After the first Zoom meeting, students again learnt three topics (or three video lessons): coordinate clauses, general adverb clauses, and adverb clauses for contrast sentences. The access to these grammar points was available right after the first Zoom meeting. The next (2 nd ) Zoom meeting to discuss the quizzes of these three topics was conducted on 15 th September.
In total, before the administration of the posttest for the research data, students had learnt at least twenty-six (26) grammar topics and had met the teacher on Zoom for six times. The length of the course for the research purpose lasted approximately one month since the pre-test from 3 rd to 6 th September with the post-test from 1 st to 3 rd September 2020. This TOEFL programme normally lasted for two months; however, due to the limited time for the research grant, a progress test was administered in the middle of the course whose results became the post-test data for the current study. (The full syllabus along with the deadline dates and Zoom schedules can be seen in Appendix D; the students could also access this syllabus prior to the commencement of the course.).
To reiterate, the students watched two types of videos: (1) video lessons that contain the explanations of the grammar topics, and (2) video quizzes that contain the explanation to the correct answers to the quizzes. All of the 60 grammar topics have already been accompanied by the video quizzes. As mentioned, each of the 60 grammar topics has its own 10-item quiz. All of the video quizzes for these 10-item quizzes have also been uploaded. However, there are some quizzes that cover more than one topic, not individual topics. Unlike the video quizzes for the individual topics, not all, yet mostly, of the video quizzes for more topics have already been uploaded.

Data collection procedure and analysis
For the quantitative data, the pre-test was completed by the students on 3 rd to 6 th September, whereas the post-test on 1st to 3rd October 2020. We collected and computed the data on the 4th of October. A paired samples t-test, also called repeated-measures t-test or dependent t-test, was employed to compare students` pre-test and post-test scores (Roever & Phakiti, 2018) For the qualitative data, the interviews were conducted by phone from 1 st to 3 rd September 2020; Copyright © 2021, authors, e-ISSN: 2502-6747, p-ISSN: 2301-9468 the questionnaire was also available on the LMS within the same dates. We analysed both of the interview recordings and questionnaire data from the 3 rd and 4 th of September.
For the data analysis of the qualitative data, a thematic analysis was employed. First, the interview recordings were transcribed. The data or the transcripts were then scrutinised for meaning units (or constructs) (Bryman, 2016). The constructs in this case refer to the different topics, or themes (e.g., internet issue, time flexibility), based upon the findings. The topics, however, were taken from not only the interview findings but also the questionnaire data. The reporting of the constructs in the research paper was categorised based on the research questions as the main themes.

FINDINGS
Based upon the proposed research questions within this study, the findings first discuss student achievement and it then moves to students` overall perceptions of the course, and their perceptions of the online flipped course.

Student achievement
To answer the first research question regarding students` performance after attending the course, students` pre-test and post-test scores were compared. The following table shows that the students appeared to perform better in their post-test as indicated by the difference in the mean scores. The mean score of the pre-test was 20.22, whereas that of the post-test was 23.62. Based upon the descriptive statistics, students appeared to have higher scores in their post-test. Albeit the mean score of the post-test was higher, the standard deviation showed that the post-test scores were slightly less homogenous. To determine the statistical significance of the pre-test and post-test scores, we computed the paired-sample t-test on the IBM Statistical Package for Social Sciences (SPSS) program (version 23). The result showed that a significant difference was found between these two tests (t(64) = 5,31, p =.001) with a medium effect size (Cohen`s d = .592). Thus, it could be concluded that students could improve their grammar achievement within a fully online flipped course after studying around twenty-five grammatical points for one month.

Students' overall perceptions of the course
To answer the second research question regarding student overall perception of the course, the participants were asked about their perception toward the course in general. This question was addressed not only to students attending the interview but also to those who filled in the questionnaire.

Positive satisfaction
Our findings showed that all sixty-four students from both the interview and questionnaire data felt that they found this course to be highly beneficial to improve their grammar knowledge or TOEFL scores. Most students mentioned that the video lessons contained simple yet detailed explanations that were easy to comprehend. One student said that "this class is very helpful to help improve TOEFL skill. The video lessons and quizzes with answers and explanations were easy to understand". "In each video, the explanation is detailed yet simple", said another student.
In addition to the simple explanation, other students mentioned that the instructional materials, or the grammar topics, were well-structured and systematic. One commented that "the teaching materials were structured systematically". Another said that "the arrangement of the teaching materials was extremely effective". One student also mentioned that the difficulty level of the quizzes became more and more challenging; thus, it helped the students comprehend the materials effectively. Thus far, these findings showed that students perceived the course to be helpful to help them improve their TOEFL skill. They also expressed that they found the course materials easy to absorb. Furthermore, the students also stated that they liked the syllabus of the course as it is well structured. The syllabus in this course refers to the list of grammar topics that they learnt in this course. All these positive comments indicated students` overall positive perceptions of the current course implementing flipped learning.

Time flexibility
As this course also utilised asynchronous learning, the students considered the learning system to be flexible in terms of location and time for them to watch the video lessons and complete the required quizzes. Although the course had deadline dates for each of the quizzes, students were still flexible to complete the quizzes in their convenient time, for they had around 3 days to complete several quizzes. A student stated that "unlike the Zoom meetings, the asynchronous learning makes this course very flexible". The student added that "there are no Copyright © 2021, authors, e-ISSN: 2502-6747, p-ISSN: 2301-9468 specific hours for us to access the materials and complete the quizzes". Some students, however, expressed that the deadlines were very hectic, making them rush to complete the quizzes. As a result, they sometimes did not have enough time to fully watch the video lessons along with the videos for the quiz explanations. A student commented that "this class is amazingly nice, but I have a problem with my time and work. I could not fully concentrate on following this class because I must complete the quizzes in a rush". Nevertheless, some students felt that the hectic deadlines motivated them to be disciplined and focus more upon their study. "Hectic deadlines trigger student motivation to complete the quizzes on time", said one student. Another stated that "this class surely motivates us to be disciplined".
Hence, the hectic deadlines for some students make them feel challenged and motivated to complete the quizzes within a short period of time. However, others may perceive this to be rather negative because they had to rush to complete all the quizzes, and this condition may impact their understanding of the material. In other words, these types of busy students might not devote enough time to learning all of the video explanations of the materials that were supposed to help them complete the required quizzes.

The use of different learning platforms
Some students expressed their appreciation for the course for the use of different learning platforms (synchronous and asynchronous learning). A student mentioned that "this course is very good and is different from other courses that I have previously attended. This class uses several learning platforms (e.g., Zoom, LMS, PDF, video) that really support the smoothness of the teaching and learning processes". Another also appreciated "the combination of synchronous and asynchronous learning".

Students' perceptions of the online flipped course Interactive feature of the synchronous meeting
It is important to note that this fully online course, the site of the current study, in the past did not use any video conferencing, and all the video lessons and video quizzes with answers and explanations were fully accessed via the LMS (asynchronous). The students and teacher, thus, never met on any live video conferencing. For the purpose of the current study, we introduced a synchronous meeting via Zoom. The Zoom meetings covered the explanations to the answers of the quizzes yet did not cover the materials. This is what makes this course to be considered an online flipped class. As a note as well, most of the explanations to the answers of the quizzes have already been provided on the LMS.
When asked about the introduction of the synchronous meetings via Zoom as a new feature in this fully online course, most of the students stated that they liked Zoom because of its interactive nature. Some students expressed that they felt motivated because they could meet the teacher directly and can see other students who are also learning hard to improve their TOEFL scores. With this interactive feature, students could ask questions directly to the teacher and could have direct clarification.
Regarding the interactive advantage, one student said that "for me Zoom is very vital and must always be used because only via Zoom there is simultaneous interaction with the teacher". Another also expressed that "I feel more motivated if Zoom could be maintained. Zoom allows students to have direct interaction with the teacher and can ask questions directly". "The Zoom meetings are like face-to-face meetings, thus allowing interaction with other students", said one student.
With Zoom, some students expressed that they could learn more because sometimes the teacher`s explanations are more detailed when students asked for more clarifications. Others also mentioned that not all questions could be easily expressed via texts, and Zoom meetings could provide a solution to this, for students could directly ask questions. A student mentioned that "via zoom we could have direct interaction with the teacher and could ask for clarification if there is something that is not yet clear".
Still related to the interactive feature, one student mentioned that the synchronous meeting could create an emotional bond between the teacher and students. The student said that "the zoom meeting must be maintained. This is where the teacher could build heart-to-heart communication with the students. The emotional bond could help students master the lessons". The student who expressed this view was a schoolteacher himself.

Internet issue and Zoom schedule and duration
Although we can say that all of the students like the introduction of the synchronous meeting via Zoom, some expressed their concern over the internet connection issue. A student said that "because of internet data, I prefer watching the videos on the LMS to attending the Zoom meeting". Another also said that "because of slow internet issue, I couldn`t fully follow the Zoom". Although the Zoom meeting was not obligatory for students to attend, yet some of the explanations of the quizzes on Zoom has not been provided on the LMS. Thus, we uploaded all of the Zoom videos, so that the students who could not attend the Zoom meeting could also access the videos.
In addition to the internet issue, some students stated that they sometimes had difficulties attending the Zoom meeting because of their hectic schedule Copyright © 2021, authors, e-ISSN: 2502-6747, p-ISSN: 2301-9468 with their work outside the course. Thus, it shows the disadvantage of synchronous learning in terms of time flexibility. A student commented that "the Zoom meeting is not flexible. It`s like attending formal schools". However, it is important to note that students` issue with time flexibility was not only for the zoom schedules (synchronous) but also for the quiz deadlines on the course LMS (asynchronous).
Others also expressed concern over the long duration of the zoom. The duration of the first Zoom meeting was very long; it lasted around more than two hours. It was because of the large number of quizzes that was covered. For this reason, we tried to minimise the Zoom duration by covering only the quizzes that tested more than one, rather than, individual topics. A student stated that "the zoom meeting takes too much time for the explanation of the individual quizzes. I hope that we only discuss the quizzes that are for more than one topic".
Thus, these findings showed that not all students might be able to attend the synchronous meetings because of internet issue and their schedule outside the course. Moreover, student also recommended that the zoom meetings should not last for a long time.

Zoom for quizzes only or lessons as well
We asked students' perceptions whether the Zoom meetings should only cover the video quizzes or also the video lessons. If both were to be covered, the course would require more time. However, covering both might offer students a better understanding because it could provide more opportunities for the students to ask questions. Most of the students preferred having only quizzes on Zoom because of the time duration. One student said that "for time efficiency, the Zoom meeting is better only for the quizzes because I have other schedules outside the course to attend to". Another stated that "it is enough to have quizzes only as it will save both time and internet data". Most of the students did not support if the Zoom meetings also covered video lessons, for all the video lessons had already been uploaded on YouTube that could be accessed on the LMS.
Nevertheless, a small number of students felt that if video lessons were also covered via Zoom, they would have more time and opportunities to ask questions, possibly leading to better mastery. One student said that "I agree if the Zoom meeting covers both the lessons and quizzes because the more exposure to the lessons, the easier for us to remember and understand the TOEFL quizzes". Another mentioned that "Zoom must also cover the lessons so that we could directly ask questions, while on the LMS or YouTube we have to write our questions and wait for the answers".

Full synchronous or asynchronous, or a blend of the two
We asked students to choose between three options: (1) fully asynchronous learning, that is, all video lessons and quizzes are accessed only via the LMS with no Zoom meetings, (2) fully synchronous learning, that is, all the lessons and quizzes are only covered via Zoom and no access to LMS and quizzes are sent in PDFs, and (3) a blend of the two types of learning. Of the sixty-four students (interviewees and questionnaire respondents), around 71% opted for the mixture of the synchronous and asynchronous learning, 19% wanted the class to be fully synchronous, and the rest preferred the class to be fully synchronous.
Thus, most of the participants expressed that they tended to prefer the use of both synchronous and asynchronous learning. It indicates that students preferred the implementation of the online flipped learning, combining the two learning modes.

Student improvement
Based on the statical computation, our findings showed that the students had significantly higher scores in their post-test than their pre-test. In other words, the students had improved their grammar scores after studying for approximately one month within this online flipped course. This aligns with other studies that have also shown that online flipped learning could lead to improved learning outcome (Stöhr et al., 2020;Swart & MacLeod, 2020). This finding can then be considered to have supported Marshall and Kostka`s (2020) proposal over the potential of the flipped learning approach to be implemented in a solely online course.
Not only did it lack participants for generalisation, but this study also lacked a control group. Thus, the results of our quantitative data need to be interpreted with caution. Although our statistical analysis showed a medium effect size, we could not conclude that it was the introduction of the flipped format as the only factor that had helped students improve their grammar scores. To come to this conclusion, further research needs to have at least a control group that does not receive the treatment of the online flipped format, for either synchronous or asynchronous could improve students' grammar achievement (Shintani & Aubrey, 2016).
A speculation could also be made that students` higher scores in the post-test were caused by students` unfamiliarity taking an online grammar test as a confounding variable. Our response to this speculation is that we had previously conducted a study in this course in which we utilised two different pre-tests, yet we found no statistical difference. This could be because of the fact that Indonesian students generally have been familiar Copyright © 2021, authors, e-ISSN: 2502-6747, p-ISSN: 2301-9468 with online grammar tests that are particularly in the form of multiple-choice items. Taylor et al. (1998) reported that there is no significant effect on students' TOEFL scores because of computer familiarity.
Furthermore, we could have expected much higher scores in students' post-test had we employed a progress test rather than a proficiency test as our post-test. That is, our post-test was an authentic TOEFL ITP test, a proficiency test, which does not only test what students have learnt within our course, yet it also tests other materials that students have not learnt. Had we employed a progress test; we might have found a higher effect size.

Students' perceptions
Based upon students` feedback, we found a number of reflections that could be considered to be suggestions to effectively implement the flipped approach in a fully online grammar course. First, students showed their positive satisfaction if the learning materials or grammar topics are systematically structured and can be accessed before the start of the course. In other words, it seems essential for the teacher to provide the syllabus prior to the commencement of the course.
Still related to the syllabus, students express positive feedback as they feel challenged if the arrangement of the grammar topics is based on the difficulty level. Students may not be well motivated to complete the materials or quizzes if they find the tasks too easy or too difficult.
Regarding the importance of having appropriate arrangement of the grammar materials, there have been some studies that have reported predictable stages of learner grammar development (Lightbown & Spada, 2013;Jia & Fuse, 2007). If the syllabus is not well structured, students may face certain materials that require understanding from more basic grammar topics. For example, for students to acquire `passive voice`, they will have to first learn about `copula be` (e.g., am, is, are, was were). Therefore, it shows the importance of the systematic arrangement of the grammar syllabus.
Second, our finding showed that students appreciated the use of different learning platforms (e.g., LMS, YouTube, Zoom, PDF) and a combination of synchronous and asynchronous learning in their online class. This finding aligns with Marshall and Rodriguez-Buitrago` (2017) research. The students commented that they sometimes needed the materials in PDF or slides because they might have difficulties accessing the video lessons. Thus, it shows that the teacher should be flexible in choosing different learning platforms that potentially could ease student learning.
Third, the students in our course mentioned that the synchronous meeting via Zoom should be maintained in that Zoom could allow students to directly interact with the teacher and other students. With this direct interaction, students felt that they could directly ask questions with simultaneous answers from the teacher. They also have the opportunity to clarify or ask for more explanations provided that the teacher`s answers are not immediately clear to them. Jauregi and Bañados (2008) found that synchronous meetings could create a pleasant virtual environment for the teacher and students that could lead to increased motivation.
Lastly, although students liked the introduction of the synchronous meeting to the course, they preferred having only video quizzes on Zoom rather than video lessons. Students expressed that they preferred shorter Zoom meeting duration. The reason for this was that they found it less flexible with their schedule outside the course. Although students preferred short Zoom duration, they in general regarded time flexibility as an edge to this online flipped course, for it offered asynchronous learning in addition to the synchronous meeting. That is, the video lessons and quizzes can all be accessed via the course LMS.
Regarding time and location flexibility as an advantage to this approach, Wang et al. (2018) mentioned that with flipped learning students can watch and replay video lessons at their own pace. Hung (2015) also reported that students spend more time learning the materials at their home, thus indicating that the flipped approach with its asynchronous feature could facilitate student autonomy.

CONCLUSION AND LIMITATIONS
Upon the first research question, we found that students` grammar achievement could improve significantly within one month after attending an online flipped learning. As a limitation, our study employed a pre-experimental design without any control group; thus, we are cautious to come to the conclusion that it was the introduction of the synchronous meeting that led to students' better achievement. We think that without the virtual meeting, the students might still have improved their grammar achievement in that they could asynchronously access all the video lessons and quizzes on the LMS. Despite this limitation, we argued that our study has confirmed the potential of online flipped learning that merits further investigation. Further research could compare student achievement in such different contexts as fully asynchronous, online flipped, and fully synchronous classes.
Regarding students` perceptions, at least we found five general themes that could become insight for online language teachers who may also want to implement the flipped learning approach. First, students expressed that a well-organised syllabus that could be accessed before the commencement of the course could raise student interest, a factor leading to positive satisfaction to the course. Second, students seem excited if their online course utilises several learning platforms that could facilitate and ease their learning experience. For example, students need the materials in the form of slides or PDFs, in addition to videos, for slides or PDFs could help them easily review the materials that they have learnt through video.
Third, we found that students recommended that the synchronous meeting in our course should be maintained, for they could utilise this meeting for direct question and clarification with an immediate answer from the teacher. The meeting was also considered interactive, and it appeared to be able to increase students` motivation as they could directly meet not only their teacher but also other students who were also studying hard for the TOEFL test. Lastly, although students supported this virtual meeting, they mentioned that the duration should be limited based on students` and teachers` needs. Seemingly the main reasons for this were students` hectic schedule outside the course and internet connection. Related to this, as a note the virtual meeting was not obligatory to the students, and all the videos after the meeting would be available for access on the LMS.
Related to our findings of the students' perceptions, there are some issues that need emphasising. First, from our interview and questionnaire data we found that all students felt positive toward the course. We might have expected different results had we also evaluated the perceptions of the students who withdrew from the course. Second, as this was an autoethnography study, that is, it was the teacher himself who acted as the researcher and conducted the interviews, the students might have provided positive comments that they think the teacher wanted to hear. Thus, our findings need careful adaptation for practical use. Further research investigating students' perception of an online flipped course needs to address all the limitations of the current study for more credible results.