The effect of e-portfolio assessment on EFL vocabulary learning and retention

Vocabulary is one of the main components in learning a new language which provides the ground for language learners to learn and use the language. In this regard e-portfolio, as a recent and novel assessment technique, can have the potential for vocabulary development. This study aimed at investigating the effect of portfolio assessment on Iranian EFL guidance school learners’ vocabulary learning and retention. To this end, 92 guidance school students in the seventh, eighth, and ninth grades, were selected as the participants of the study. They were randomly assigned into two experimental and control groups. While the control group followed the conventional class quizzes the experimental group practiced e-portfolio assessment. The participants in the experimental group were asked to create their e-portfolios and keep a record of what they learned during and after the online sessions. They were also asked to include the reflection sheets in their e-portfolios. Three parallel tests as a pre-test, an immediate post-test, and a delayed post-post were used to gather data about the effect of portfolio assessment in each grade (a total of nine tests). The results of a one-way ANCOVA revealed that the participants of the experimental group outperformed the participants of the control group in terms of EFL vocabulary learning and retention. Considering the outcomes, the study presents some implications for practitioners including language teachers, curriculum and course developers, and language learners.


467
Focusing simultaneously on the process and product of learning alternative assessments require learners to create, do or produce something in authentic contexts while being assessed according to that performance (Brown & Abeywickrama, 2019). Recently, in response to the shortcomings of traditional assessment, different kinds of alternative assessments such as portfolios, journals, interviews, observations, self-assessments, and peer assessments have been used. Among all the alternatives in assessment, portfolio and e-portfolio assessment have gained considerable attention and popularity among teachers, course developers, and other stakeholders.
As Moya and O'Malley (1994, p. 2) mention, "a portfolio is a collection of student's work, experiences, exhibitions, self-ratings", while "portfolio assessment is the procedure used to plan, collect and analyze the multiple sources of data exist in the portfolio" (p. 2). Different kinds of portfolios have been introduced and used in educational contexts. The typical paper-based portfolios focusing on the outcomes seem to be the most applied one; however, e-portfolio is the most recent and novel technique with immense potential.
An e-portfolio is defined as "a digitized collection of artifacts including demonstrations, resources, and accomplishments that represent an individual, group, or institution" (Lorenzo & Ittelson, 2005, p. 2) which reflects "accomplishments and learning" and proves "an individual's learning and additionally capacity" (Gray, 2008, pp. 6-7). The key features in the definition of an e-portfolio are "being digital, organized, searchable, and transportable" (Yastibas & Yastibas, 2015, p. 5).
E-portfolios as one of the main developments of technology in education help learners gather their work in an organized, accessible, and transferrable way (Rhodes, 2011). They improve both the learners' development and their active roles in learning (Goldsmith, 2007). E-portfolios improve learners' understanding and awareness of the learning processes and help recognition of the idea that learning has to also occur outside of the classroom, which can promote their autonomy (Gonzalez, 2009). E-portfolios demand learners to carry out some self-evaluation. Therefore, they provide learners with helpful and guiding feedback (Reese & Levy, 2009). They improve the reflection of learners by helping them understand the development they have had and record the "evidence of their capacity for critical thinking, analytic reasoning, and integrative learning" (Rhodes, 2011, p. 5).
Teachers and course developers of language programs have utilized portfolios and e-portfolios in various academic contexts. Researchers have also paid particular attention to these two assessment techniques in recent years (e.g., Afrianto, 2017;Barrot, 2021;Gan & Lam, 2020;Halim & Lestari, 2019;Hyland & Hyland 2019;Ismailov & Laurier, 2021;Kusuma, et al., 2021;Lam, 2018;Lam, 2019;Mphahlele, 2022;Ngui, et al., 2020). Therefore, there are a number of empirical studies on the effect of portfolio in general and e-portfolio in particular on vocabulary learning. Berimani and Mohammadi (2013), and Nassirdoost and Mall-Amiri (2015) found the positive effect of the portfolio on vocabulary learning of EFL learners. Further, Zarei and Baftani (2014) compared the effects of different techniques of vocabulary portfolio including word map, word wizard, concept wheel, visual thesaurus, and word rose on L2 vocabulary comprehension and production. The findings revealed no significant differences among them in the vocabulary production rate of the learners. Sharifi et al. (2017) found the positive effect of e-portfolio on the vocabulary learning of their EFL participants. Their findings reflected that the e-portfolio motivated the learners to learn new vocabulary items because the participants reported that they enjoyed keeping the e-portfolio and benefited from integrating technology with their educational activities. Their study does not follow up on the retention of the developed vocabulary items, but the authors share some practical guidelines on having e-portfolios in the EFL curriculum.
It seems that EFL learners in Iran sometimes face difficulties in using appropriate and sufficient lexical items in different situations. This problem is usually worse among Iranian EFL school students. This can be due to several reasons but following old and traditional ways and methods of teaching and assessing seems to be the biggest one. English language instruction in Iranian schools is a permanent object of criticism, and most of the stakeholders of education, including teachers, students, and even parents are dissatisfied with the nature and quality of the English language teachinglearning process in schools, which is under the influence of traditional methods and approaches. Following the present curriculum, which gives little or no attention to the real performance of learners and often leaves them with no information or guidance about how to use what they learn, can demotivate them and lead to an ultimate disappointment which, in turn, may slow or stop their language development. Therefore, it is time to practice new and confirmed ways of fostering and assessing learners' performance and development. Portfolio assessment seems to be one of the most popular and effective techniques in this regard. It should be mentioned that no studies (up to this date) were found to investigate the effect of e-portfolio assessment on vocabulary learning and retention of Iranian EFL learners in the context of school. Therefore, the present study will be carried out to investigate this potential effect. Based on the intentions of this study and the problems mentioned, the following questions and hypotheses are presented: RQ1: Does e-portfolio assessment have any significant effect on EFL vocabulary learning of Iranian guidance school students? RQ2: Does e-portfolio assessment have any significant effect on EFL vocabulary retention of Iranian guidance school students?

METHOD Design of the Study
The study applied an experimental design with control and experimental groups following pre-test, treatment, immediate post-test, and delayed post-test format. To ascertain the effects of e-portfolio assessment on the learners' vocabulary learning and retention, the participants were tested before and after the instruction. There was a delayed post-test as well to investigate the retention of the vocabulary items. The dependent variables were EFL vocabulary learning and retention, and the independent variable of the study was e-portfolio assessment.

Participants
The twelve-year educational system of schools in Iran is divided into three major parts of a six-year elementary school, a three-year junior high school, and a three-year senior high school. The elementary school does not entail any English language courses and students have their first English course in the seventh grade at junior high school. Therefore, the focus of the present study was on junior high school students. The participants of this study were 93 EFL learners studying English as a foreign language at seventh, eighth and ninth grades of junior high school in Iran. To form the control and experimental groups the participants were randomly assigned to two study groups.

Data Collection Instruments
To gather data from the participants of the study, the researcher used three parallel tests for each grade (seventh, eighth, and ninth). The first test was a teacher-made pre-test including three parts with 15 matching items, 10 multiple-choice items, and 10 cloze items. Then two parallel tests were designed and used as the immediate and delayed post-tests. The vocabulary items presented to the participants during the treatment phase were selected from the Prospect series the English course books of guidance schools in Iran designed and published by the Ministry of Education. Prospect One was the course book for the seventh grade, Prospect Two for the eighth grade, and Prospect Three for the ninth grade. This study gathered the data using nine tests based on the educational grades. There were three parallel vocabulary tests of pre-test, immediate post-test, and delayed post-test for each grade (the seventh, eighth, and ninth). Three experts checked the validity of the tests and approved validity after the required modifications were applied. A pilot study was conducted to estimate the reliability of the tests. The reliability indices of all the nine vocabulary tests fall within the acceptable range, the results of which are presented in Table 1. Since the study was conducted to assess learners' performance using e-portfolio, the researcher used "WhatsApp" and "Shad" messengers to receive and share the work of participant students. The learners created, edited, and stored their e-portfolios using computer and mobile platforms.

Data Collection Procedures
At the outset of the study, the participants were randomly assigned to control and experimental groups. Each group included three levels of seventh, eighth, and ninth grade students following the syllabus particular to their grade.
A vocabulary pre-test was used to confirm the homogeneity of the paired groups. The study was conducted during the Pandemic and the educational system of the whole globe including Iran had turned to virtual online classes. The participants of the control and experimental pairs had the same course books for each grade and were provided with the same content and vocabulary items presented in the books. The experimental group, however, was fully introduced to the concept of portfolio and was informed on the assessment procedure using their course portfolios. The control group received the same instruction but did not practice keeping a portfolio and for assessing their learning the typical and traditional method short quiz prevalent in the educational system of Iran was used. As the instruction initiated, each online session learners received new vocabulary items from their course books (Prospect 1, 2, and 3). The teacher, who was one of the researchers, provided the students with extra and authentic materials (videos, audios, pictures, etc.). Each session learners reviewed whatever they learned in the class, and the teacher explained to them how they could use the instructed items in real communication contexts.
At the end of each online session, the teacher asked the experimental group to find or create any related materials or items (video or audio files, pictures, flashcards, shapes, objects, newspapers, magazines, etc.) on the instructed items of the day. In the next session, students presented their found or created materials and received comments from the class members. Then the teacher provided them with feedback. Later the teacher reviewed some of the portfolios to supervise the quality of the ongoing process. The participants in the experimental groups were asked to include any material they had created, found, or learned in their e-portfolios. Later they were also asked to review their e-portfolio entries, select the appropriate ones, and edit them if required. They also included their own reflections or any feedback they received from their parents, classmates, and the teacher. The treatment lasted for 10 online sessions.
After the treatment, the learners took the immediate post-test to measure the effectiveness of the instruction. After a three-week time interval, the delayed post-test was used to measure the retention of the instructed items.

Data Analysis
Version 25 of Statistical Package for Social Science (SPSS) software was applied to analyze the gathered data. To investigate the effect of e-portfolio assessment on EFL learners' learning of vocabulary items a one-way ANCOVA was conducted. Moreover, a second one-way ANOVA was run to find out the effect of e-portfolio assessment on EFL learners' vocabulary retention over time.

FINDINGS
The present study was undertaken in order to explore whether e-portfolio assessment has a significant effect on Iranian junior high school students' vocabulary learning and retention. It used one-way ANCOVA for analyzing the results of the tests. According to Mackey and Gass (2005), there are times when a preexisting difference (covariate) could be seen in the groups of a study, and it should be controlled. The adjustment depends on the greatness of the difference between groups and the change between the pre and post-tests. The pre-tests and immediate post-tests of groups at each level were compared to investigate the effectiveness of the e-portfolio assessment process. Therefore, the results of the immediate post-test and delayed posttests of groups were compared to examine the retention of the items which were taught.
Before conducting one-way ANCOVA for the first question, the assumption of a linear relationship between the dependent variable and the covariate was checked by the scatterplots (Figure 1):

Figure 1 Relationship between Pre-Test Scores and Vocabulary Learning Scores
Finally, the assumption concerns the relationship between the covariate and the dependent variable which was checked by the homogeneity of the regression slopes. The related results are presented in Table 2. Table 2 shows the significance value or probability value (.06) as safely above the cut-off. Since the significance value for the interaction is greater than .05, the interaction is not statistically significant, indicating that the assumption is not violated. . The actual difference in the mean scores of the immediate post-test between the control group in comparison with the experimental group is quite large. Descriptive statistics of the immediate posttest are presented in Table 3.  Table 3 reflects that the mean score for the control group (M=26.18, SD=4.82) was higher than that of the experimental group (M=33.11, SD=3.32). As previously mentioned, after checking out the assumptions a one-way between-groups analysis of covariance (ANCOVA) was conducted to find out whether the difference in the mean scores was significant or not and to investigate the effectiveness of e-portfolio assessment on EFL learners' vocabulary learning. The independent variable was the effect of e-portfolio assessment, and the dependent variable consisted of scores on the test after the intervention was completed. Participants' scores on the pre-invention administration were used as the covariate in the analysis. Table 4 shows a significance value of .28, which is much larger than the cut-off of .05. Therefore, the assumption of the equality of variance is assumed. As a result, the error variance of the dependent variable (immediate post-test) is equal across groups.  The second ANCOVA was run to investigate the effect of e-portfolio assessment on vocabulary retention. But before carrying out one-way ANCOVA for the second question, the assumption of a linear relationship between the dependent variable and the covariate was checked by the scatterplots. The related results are presented in Figure 2.

Figure 2 Relationship between Pre-Test Scores and Vocabulary Retention Scores
Furthermore, the assumption concerns the relationship between the covariate and the dependent variable which was checked by the homogeneity of the regression slopes, the results of which are shown in Table 6. Table 6 shows the significance or probability value as .06, being above the cut-off. And since the significance value for the interaction is greater than .509, the interaction is not statistically significant, indicating that the assumption is not violated. A one-way between-groups analysis of covariance (ANCOVA) was conducted to investigate the effectiveness of e-portfolio assessment on Iranian advanced EFL learners' vocabulary retention by comparing the delayed posttest scores of the experimental and control groups. The independent variable was the effect of eportfolio assessment, and the dependent variable consisted of scores on the test after about a threeweek intervention. Participants' scores on the preintervention administration were used as the covariate in the analysis. Descriptive statistics of the delayed post-test are presented in Table 7. As Table 7 indicates the mean score for the control group (M=25.95, SD=5.49) was higher than that of the experimental group (M=33.08, SD=3.41). The results of the ANCOVA test would reveal whether the difference in the mean scores of the delayed post-tests in the two study groups was significant or not. Participants' scores on the immediate post-test were used as the covariate in the analysis. Table 8 shows a significant value of .39, which is much larger than the cut-off of .05, indicating that the assumption of the equality of variance is assumed. Therefore, the error variance of the dependent variable (immediate post-test) is equal across groups.  Table 9 below reflects the difference in delayed post-test scores of the experiment group (M=33.08, SD=3.41) and control group (M=25.95, SD=5.49) is statistically significant (F=183.47, p≤0.05) indicating that e-portfolio assessment had a positive effect on vocabulary retention of the participating EFL learners.

DISCUSSION
In spite of the difficulties in implementing eportfolios (Poole et al., 2018), they have been gaining increasing attention and are being widely used in EFL contexts (Hinojosa-Pareja et al., 2020;Oh et al., 2020). The positive effect of using eportfolio assessment has been mentioned in some works. As Goldsmith (2007) notes, e-portfolios can improve the learning process and develop an authentic assessment. He also states that practitioners can benefit from e-portfolios because they help them to understand "how well they are educating their students" and they are able to evaluate "their language program's experiences and accomplishments, and their relationship with educational objectives" (p. 31). E-portfolios can also improve learners' understanding and awareness of the learning processes and help the recognition of the idea that learning has to also occur outside the classroom which in turn can encourage learners' autonomy and make them more responsible for their learning (Gonzalez, 2009). According to Pardo (2021), the digital nature of the e-portfolios makes storing the students' work samples in different contexts and analyzing them in time periods possible, which makes the nature of the work much more learner-centered and motivating.
Considering the treatment procedure in the present study, learners in the experimental group were provided with acceptable practices, feedback, and learning reflections. They were, furthermore, asked to collect their work and write reports on their performances which in turn demanded an active engagement of learners in the process. All of this made the retention process easier for them because they were actively involved in their learning and turned to be more autonomous in their development. The learner-centered nature of the portfolios can promote learner autonomy (Torabi & Safdari, 2020). Furthermore, e-portfolios are effective in improving learners' goal-setting, planning, monitoring, as well as their development (Pardo, 2021) because learners become managers of their learning environment (Namaziandost & Çakmak, 2020). As Rhodes (2011) mentions, e-portfolios can motivate learners since they present and represent their work, which in turn can make them more active learners. Accordingly, the participants who received the eportfolio assessment displayed better performance in recalling the vocabulary items than the learners who received the traditional quizzes which can be, in part, related to the self-reflection nature (Namaziandost, Sawalmeh, et al., 2020;Yastibas & Yastibas, 2015) of the e-portfolios. Reflecting on one's own learning and being more engaged in it improves learning to a considerable extent (Ahn, 2004;Schmitz et al., 2010).
Therefore, it can be said that assessing learners by e-portfolios had a positive effect on EFL vocabulary retention of learners.
The results of this study are in line with the study of Berimani and Mohammadi (2013), Nassirdoost and Mall-Amiri (2015), Sharifi et al. (2017) and Pardo (2021) confirming the positive effect of portfolio assessment and e-portfolio assessment on vocabulary learning of EFL learners which can be attributed to the active, yet self-paced autonomous learning occurring while developing and holding academic portfolios. There are, however, no similar studies concerning the effect of e-portfolio or even portfolio assessment on EFL vocabulary retention of the learners.
There are several factors influencing vocabulary retention in learners. As Mayor (2014) mentions the information which is put in storage in the sensory memory, is transmitted from the shortterm to the long-term memory when sufficient attention and time are provided. The new information is learned in an association progression (Mayer, 2005). If the information is treated actively being used in meaningful interactions, it is precisely transmitted from working memory to long-term memory (Schmitt, 2000). Keeping an e-portfolio made the participants of this study responsible for their own learning by making them pay extra attention to the vocabulary items. This happened in meaningful use of the words in archived portfolio samples by easily updating the multiple media (Namaziandost, Hosseini, et al., 2020), and spending sufficient time to collect, prepare, and present the gathered samples due to the self-based assessment nature of the e-portfolios which lead to learner development (Namaziandost, Razmi, et al., 2020). Welsh (2012) asserts that learners' active participation in their assessment process enhances their higher-order skills such as analysis, synthesis, and evaluation. All of these helped in storing the vocabulary items in long-term memory and fostering retention.
Using e-portfolio assessment as an ongoing assessment process was an effort to assess learners' real performances, expand their vocabulary domain, extend the retention time, make them more autonomous in learning and self-assessment, inform them about what they have learned and how they can use them in different situations, and eventually motivate them while making learning a fun and satisfying experience for them. Considering the beneficial effects of e-portfolios, introducing them to Iranian EFL context with a long history of traditional educational system depending mainly on summative assessment (Firoozi et al., 2019;Namaziandost, Rezvani, et al., 2020) seems highly crucial.

CONCLUSION
The present study attempted to answer two research questions on the effect of e-portfolio assessment on EFL vocabulary learning and retention of those learners. The findings of the study revealed a significant difference between the immediate posttest results of the two study groups indicating that the e-portfolio assessment had a significant positive effect on vocabulary learning of EFL learners. Furthermore, the participants of the experimental group outperformed the control group in the delayed post-test proving the significant positive effect of the e-portfolio assessment on vocabulary retention of EFL learners as well.
The study may have beneficial contributions since it made learners aware of what they have learned and how they can improve their learning by extending their vocabulary knowledge and making them more responsible for their education. Keeping an e-portfolio also helps learners expand the acquired items to other situations, and gives them opportunities for further learning. The study may also be helpful because e-portfolio assessment potentially increases learners' motivation and autonomy, and eventually encourages them to learn more and better in the ongoing development process and even later.
The findings can have some pedagogical implications for various academic practitioners. First of all, the researcher observed a great enthusiasm and a positive reaction toward using eportfolios as a way of learning lexical items on the side of the learners. They were willing to gather their work samples, give and receive feedback and share them with other students. Since several studies, including the present study, have proven the positive impact of using this technique, teachers can put more creativity into their work and use eportfolios as an alternative way of assessing learners' vocabulary development.
Furthermore, material writers, curriculum developers, and syllabus designers can take the advantage of this assessment as a movement from the traditional ways of teaching and assessing the English language at schools to more novel and confirmed ways. E-portfolio assessment undoubtedly is one of the most useful and practical teaching and assessment techniques, and decisionmakers can give teachers more freedom to use effective and modern methods of teaching in their classrooms, including portfolios and e-portfolios. This study benefited from e-portfolios involving a lot of interaction and communication between learners and teachers with no need for much inperson interaction. Considering the present limited educational conditions resulted by the Covid-19 pandemic, a drastic schedule shift is required in academic contexts to meet the requirements of this condition. In this line, policy-makers can come up with more ideas for cyber education, including eportfolio.
Language learners can also benefit from the findings of this study. Since the new generation embraces technology and innovation more than ever before, implementing these elements in their education can strongly encourage them to adopt an active role and develop their learning. E-portfolio and more recent types of it, such as chat portfolio or online portfolio can be beneficial for learners in case of collecting, saving, sharing, and assessing their work samples.