Eliciting teachers ’ understanding and their reported practices on school-based formative assessment : Methodological challenges

Ministries of Education in many countries have adopted various forms of school-based assessment (SBA) to replace (for example, New Zealand) or complement (for example, England, Australia and Malaysia) more conventional forms of assessment such as tests and examinations. Central to these alternative approaches to SBA is formative assessment. In recent years, a body of research has been built investigating various aspects of SBA in Malaysia, but there has been a dearth of studies exploring what practising teachers believe and do regarding implementing formative assessment in their own classrooms. The present article reports some of the findings of a case study in which ten Malaysian primary school teachers of English were interviewed to identify the extent of their understanding of formative assessment and their reported practices of providing feedback in an SBA environment. Initially, the teachers revealed a general lack of understanding of the difference between formative and summative assessment. In such a situation, it would seem that the teachers are unready to implement SBA at the present stage, despite it having been mandated in Malaysian schools since 2011-12. However, later in the interview, they reported implementing various forms of feedback, some of which might be regarded as formative. There is a need, therefore, to differentiate between teachers‟ explicit knowledge and their implicit understanding of matters such as formative feedback. The inherent limitations of self-report data emerging from interviews will be discussed and how these might be overcome.


INTRODUCTION
The assessment of students" learning -an essential function in any educational setting -has been defined as "a tool for gathering useful information about teaching and learning through an orderly process of inquiry based on a set purpose that effectively informs practice and decisions" (Hasim & Tunku Mohtar, 2013, p. 3).Conventionally, largely perhaps for logistical reasons, there has been an emphasis on a posteriori assessment of students learning by such means as tests and examinations.The purpose of such summative assessment has been defined by the Organisation for Economic Cooperation and Development (OECD) as "to measure what students have learnt at the end of a unit, to promote students, to ensure that they have met the required standards on the way to earning Indonesian Journal of Applied Linguistics, 8(1), May 2018 159 Copyright © 2018, IJAL, EISSN 2502-6747 certification for school completion or to enter certain occupations, or even as a method to select students for entry in further education" (OECD/CERI, 2005, p. 1).Over fifty years ago, Scriven (1967) and Bloom (1969) pointed out that, in contrast to the judgemental nature of summative assessment, the purpose of formative assessment was to make changes in the teaching/ learning processes; in other words, assessment for learning rather than the testing of learning after it has (or has not!) occurred.According to Mathison (2010), formative assessment can be used to improve learning, to generate insights on educational issues, to promote continuous evaluation and to strengthen programs and organisations.
"Formative assessment provides information about the learning process that teachers can use for instructional decisions and students can use in improving their performance, which motivates students" (Wiliam, 2011, p. 8).But in whatever way formative assessment is defined, its effectiveness is dependent on how it is used (Cauley & McMillan, 2010).And if it is to be effective, the key agents in the process of change are the schools and teachers in which the students are learning; hence the increasing move towards schoolbased assessment.

School Based Assessment in Malaysia
In 2011, the Ministry of Education introduced schoolbased assessment in all primary schools to empower schools and teachers to conduct quality assessment of their students (Hashim, Ariffin & Muhammad Hashim, 2013).Compared to the previous reliance on summative assessment via tests and examinations, SBA was intended to be a more holistic, integrated and balanced assessment as part of the Government Transformation Plan (GTP) in the effort to produce world-class human capital (Raman & Yamat, 2014).
The Malaysia Ministry of Education (MoE) (2013) has stated clearly that SBA, functioning both as assessment of learning and assessment for learning, can be carried out summatively and/or formatively.Summative assessment is carried out at the end of each learning unit through monthly and semester tests.Formative assessment is a continuous activity carried out alongside the normal learning and teaching process, and the reporting system requires statement evidences that explain how students demonstrate what they know and can do.
To facilitate SBA, the Examination Board (2011) has defined Performance Standards as the main reference for all concerned -students, teachers and parents -to know clearly what standards are needed to achieve after a certain period of instruction, and for other stakeholders to understand the national education system"s aspiration and goals.The Performance Standard has six bands: 1, students know or can perform basic skills or provide response to the basics; 2, they are able to illustrate their understanding and interpret and explain what has been learnt; 3, students are able to use their knowledge to implement a certain skill in specific situations.4, they are able to carry out a certain skill more systematically; 5, they can implement the skill in new situations more systematically, consistently, and positively; finally, 6, students are described as being able "in an exemplary manner" to illustrate ideas creatively and innovatively, and can discuss how to systematically get and deliver further information.
The Examination Board (2011) also explains how the Performance Standard fits into SBA: teachers" preparation begins by choosing a subject content based on the Curriculum Standard Document (MoE, 2013), and planning lessons, preparing teaching materials and delivering the content using various strategies.They will then interpret students" understanding of what was taught through diverse strategies and appropriate formal or informal assessment tools (which may include worksheets, observation, quizzes, check list, report assignment, homework and tests) and will refer to the standard performance document to record students" achievement.If the students achieve the standard, the teacher may proceed with the next content; if not, the teacher will have to conduct an intervention session to guide the students appropriately (Examination Board, 2011).
In summary, from a sociocultural perspective, formative assessment is a dialogic process between teachers and students regarding the progress of individual learners, and the results reported to parents, school and education authorities.In such a dialogue, mediating tools play a vital role in providing information to enable a teacher to make an overall teacher judgement (OTJ) on a student"s learning achievement and progress.The validity of the OTJ is affected by: the inherent quality of the tools themselves; the value placed on the tools by the teacher; and the willingness and ability of the teacher to effectively manage the feedback process, which is influenced by the opportunities and constraints that occur in the specific setting in which the teacher works.However, the prime consideration is the extent to which teachers working in an SBA environment understand the nature and means of formative assessment.Clearly, the effectiveness of implementing the policy depends upon the teachers" knowledge and understanding of the principles, as well as the tools, they can use to assess their students both formatively and summatively.

Teacher cognition
Research into teacher cognitionwhat they think, know and believe -has emerged as a major area of inquiry in the field of language teaching over the past three decades, and in particular the extent to which teachers" stated beliefs match what they actually do in their classrooms.It is widely accepted that teachers" stated beliefs play a significant role with respect to instructional practices.For example, it was early recognised that, according to Isenberg (1990, p. 325) "teachers" thinking and beliefs influence instruction, teachers" thinking and decision-making influence professional growth", and that "the beliefs teachers hold influence their perceptions and judgments, which, in turn, affect their behaviour in the classroom" (Pajares, 1992, p. 307).However, although teacher cognition does have an influence on what goes on in the classroom, teachers" practices "do not always reflect stated beliefs, personal theories, and pedagogical principles" (Borg, 2003, p. 91).This is because any number of factors, internal and external to the classroom impact upon practice; for example, fixed rows of students" desks can thwart the teacher"s intention to organise group work, or the externally imposed assessment procedures may run against what the teacher believes or knows about how best to assess his/her own students.Nevertheless, teachers inescapably exert their influence on "the effectiveness of the teaching/learning process" (Griffiths, 2007, p. 91) and in this includes the provision of formative feedback.As has been noted by Hasim (2014) in a recent study that the teacher is an agent of change in which "teachers" perspectives influenced teachers" pedagogical practice that consequently affected the students" development of learning" (p.305); in this case, it was evident that an integration of formative assessment enhanced teaching and learning.In promoting positive development of the assessment system, a shift in perspectives about teaching, learning, and assessment needs to occur that consequently leads to a shift in roles and practices.Borg (2006Borg ( , 2012) ) has reported a number of empirical studies investigating teachers" beliefs and reported practices regarding assessment, but little has considered the assessment perspectives of language teachers in primary schools, at least in Malaysia.Therefore, the focus of the present study of language teachers in Malaysian primary schools occupies a so far underresearched space.
In respect to research methodology, due to the unobservable nature of teacher cognition, a crucial issue in this domain of inquiry is what counts as evidence (Borg, 2003(Borg, , 2006)).A wide range of strategies and methods have been employed to collect data in language teacher cognition research.Borg (2012) reviewed 26 teacher cognition studies published in 2011; of these, 25 used interviews as the main approach, eleven of which also used questionnaires; one study employed only interviews, and another only questionnaires.Nine of the studies also carried out observations to support interview data.The present study followed an initial questionnaire with semi-structured interviews with a sample of the responding teachers.The findings themselves were interesting, but what is more to the point is that the analysis of the data raised questions about the use of such self-report procedures to ascertain what teachers believe and do.

METHOD
This article reports some of the findings from a case study carried out in Malaysia which sought to elicit the understanding of primary school teachers with regard to formative assessment in the SBA environment in which they worked.The participants were all English language teachers working in five primary schools in a mainly middle-class suburban area on the outskirts of Kuala Lumpur; in Malaysia, students from Year 1 who are taught the curriculum subjects by specialist teachers.Initially, a survey was carried out with 47 respondents and this was followed by semi-structured interviews with volunteer teachers from the five schools.Prospective interviewees were given a letter of explanation about the project, their questions and comments were addressed at a meeting, and ten teachers (two from each school) completed consent forms agreeing to be interviewed and for the interview to be audio-recorded.For the sake of brevity, the present article reports the interview findings related to two of the research questions which guided the study:  What do the participants understand by formative assessment? How do these teachers say they provide formative feedback?

FINDINGS
The interviews were conducted in English, the teachers" (and interviewer"s) second language.The interviewees" comments reported below are verbatim: no attempt has been made to "tidy up" their syntax or lexical choices.
With regard to the first research question, most of the participants (Teachers 2, 3, 6, 7, 8 and 10) were, to a greater or lesser extent, confused about the difference between formative and summative assessment: T2: For me, formative assessment is an assessment that you do once you finish every skills or lessons.
Once you done it just assess their understandings whether it"s by written matter, orally or group work whatever that seem suitable at that time.
T3: In my mind, formative assessment is something that we tell the pupil that it"s exam and they have to do it by themselves.Not friends helping, teachers may help saying meaning of words and all, but not fully helping them.That is formative.
T6: Interviewer: So do you know something about summative assessment?
Interviewee: Summative, let me guess, something like ongoing observation?
T7: What I know about formative assessment is the test is aim to check the learning progress of the students.For example, when the teacher teaches a certain topic, then the teacher assesses at the end of the class.
T8: Formative assessment is usually non-formal kind of assessment.Usually, in Malaysia, formative assessment will be involved in major examination, so maybe in schools you call it as monthly test.
T10: Formative assessment is exam-oriented.So, we have to test.We also have monthly test.Usually, Copyright © 2018, IJAL, EISSN 2502-6747 in March and in September.Summative assessment is for the whole year.For example, we have listening, speaking, so we can do oral exam and then we can also look from their behaviour.We also give them projects or assignments that they have to finish at home.
Another teacher showed a somewhat greater awareness about formative assessment: T9: From what I know, formative is more to the topics and then the teacher will assess pupils once the topics have been taught and then they will guide students on how to achieve some assessment based on the topics that they have learned.
Only one teacher provided a relatively comprehensive understanding of formative assessment: That means if that paragraph contains five sentences, which means the second paragraph for assessment also must contain five sentences, about three to five assessments.That means we are not using the same item to assess the student, but something that is parallel.That means they don"t know that they are being assessed.It"s just like an observation to their work.See, they are working in their group, exercises that they do.Formative assessment is like the scaffolding process for the teacher to guide the students to achieve a skill.That means trying to make everybody achieve that skill to a certain level.So that is formative.That means… you know… it"s more to teachers" observation on the students" work during the classroom and how they respond to your lesson.
Obviously, T5 perceived formative assessment from a number of perspectives.It could be concluded that, in her opinion, formative assessment is an ongoing scaffolding process in which both teachers and students are involved, and it is normally implemented during class.Besides, she mentioned that it is not possible to assess all students at once, but to do so with a few students at a time in different lessons using different materials.
In conclusion, it appeared that most of the interviewees were not equipped with an explicit knowledge of formative assessment.In the light of this, it may be supposed that they were unable to carry it out effectively.However, further probing indicated thatalthough they might not have been able to "talk the talk" -they were, to some extent at least, able to "walk the walk".
The second research question explored teachers" practice of giving formative feedback to their learners.Firstly, almost all of the teachers stated that they gave oral and/or written feedback of some sort: T1: I provide feedback orally when the student gives the answer or give the suggestion or idea in oral …when it comes to written, it"s usually based on the written essay that the student has passed up.
T2: I provide both, orally and by written… It depends on how they response to me.
T3: If it"s written work, I give written.But when it"s reading or activity it"ll be oral.
T5: Yeah.Oral feedback, I usually do.I think -I believe in giving oral feedback because kids love it.I also give them written feedback, but not often.
T6: I would like to give them feedbacks in oral, but I rarely give it in written.I only feel satisfied if I can reach them by their side, correct them personally, so I need a little more energy but I preferred doing like that.
T7: Usually by oral, if they doing tasks orally and I give written feedback sometimes, for example, for their work in exercise book.
T8: For me, I provide them feedback by oral and written.
T10: By oral and by written.
Secondly, the teachers said that they used feedback for various reasons; sometimes merely to correct students" errors: T2: For example, what I did today is I discussed them the objects found in certain basis.Sometimes there will be.It may be hospitals, shops.I say "shop is not correct.Shop is not an object.A kite is an object."Some say "Air Con".I said, "it"s Air Conditioner."So, I correct them.
T3: During my lesson, I give them the feedback.I correct them.Written feedback, I correct their mistake.Like, if it"s grammar, I put a line, correct them, spelling mistake, I could correct them.
T4: I find some general errors everybody makes then I just do correction.
T7: For example, they are writing, maybe they are writing answers for questions that I give them.So, I saw that there are some mistakes that they do.So instantly I ask them, I told them "okay, you are doing a mistake there.There is a mistake, so here and here.So you need to correct like this." Others saw feedback as a means of encouraging students: Only two of the teachers were explicit about providing feedback for formative purposes: T1: In terms of written feedback, if they"ve not included in essay for example, the same question on "give me 5 tips on how to protect yourself".Let say the student… say, he has left one of the tips or suggestions, so I"ll add that into it.I"d say what the student has forgotten to say.In terms of written feedback, first of all I"ll write whatever they have left out, the students left out and I will give my suggestion and as a remedial work I"ll ask the student to rewrite whatever the student has missed out.
T8: for reading, writing, I"ll check for mistakes and then I"ll write their strength and then their weakness and ways of how to improve.I will see the corrections and maybe from there, I will give some advice … because they should know their problems, they should know their mistakes.To learn from their mistakes, we should give an advice to them.
However, most of the teachers realised that the feedback was formative in the sense of their need to reconsider their teaching: T2: Based on their worksheets …if the students are not able to do it well, that means, they did not understand the lesson well, I will consider about adding more lessons for them to improve… If the students complete it well then I will go on with my instruction.

DISCUSSION
In addressing the first research question, findings indicated that teachers are somewhat confused of the two forms of assessmentsformative and summative and it was decided to quote so many comments by the teachers so that their individual trajectories can be noted; for example, Teacher 8 initially said "formative assessment will be involved in major examination, so maybe in schools you call it as monthly test".Apparently, she was thinking about summative rather than formative assessment and confused the two.Yet, Copyright © 2018, IJAL, EISSN 2502-6747 towards the end of her interview, she showed a depth of practical understanding by explicitly referring to formative assessment as "…how [it] informs me on how to prepare better instructions for my pupils."Similarly, at first Teacher 7 confused the two terms by saying, "What I know about formative assessment is, the test is to aim to check the learning progress of the students", but subsequently said that she did practice assessing students formatively: "I saw that there are some mistakes that they do.So, instantly I ask them, I told them that "okay, you are doing a mistake there".By contrast, Teacher 5 consistently showed a greater depth of understanding than her colleagues.This supports Talib et al."s (2014) survey research that many Malaysian teachers have insufficient knowledge about assessment and that a forceful alignment between content of the curriculum and teachers" competency is truly needed in Malaysian SBA system.To add, this finding is consistent with the previous study conducted by Edman, Gilbreth, and Wynn (2010) that many teachers confused about the conception of formative assessment with various other forms of assessment including summative and authentic assessment in both understanding and implementation.According to Chappuis and Stiggins (2008), formative assessment and summative assessment serve for different assessing purposes, and whether the assessment is formative or summative depends on how the teacher use the assessment data gathered from the students.It also could be the reason that primary school teachers were accustomed to the traditional classroom assessment system, as what Raman and Yamat (2014) had illustrated in their study that, Malaysian primary school teachers have grown accustomed to traditional assessments in evaluating student performance at the primary and secondary school levels.This is understandable that before the reform of SBA, the traditional classroom assessment in primary schools in Malaysia has been conducted in a highly summative way for a long time, that is, teachers were used to assess students" learning outcomes with summative assessing purposes rather than students" learning process with formative assessing purposes.However, the new reform of SBA requires teachers to make a paradigm shift from assessing student learning outcome only to assessing both student learning outcome and student learning process.In case of that, if a teacher did not equip with adequate knowledge of what FA is and how to implement FA, the teacher will likely practice FA in ineffective way or even decline their FA practices in classroom and continue the summative mode of classroom assessment which he or she was accustomed to implement before, as Taras (2007) had illustrated in his study that the conceptual confusion of assessment for learning (AfL) with assessment of learning (AoL) results in teachers being less likely to adopt AfL as a strategy in their classrooms.
In relation to the second research question to address on how teachers provide formative feedback, findings of this study indicated that majority of teacher respondents provided feedbacks by oral and by written, however, they were accustomed to do corrections towards students work only, or give students praises and encouragements as feedbacks, and they seldom provided advices on what students can do to improve.In addition, majority of teacher participants were used to provide feedbacks to the whole class or by groups, rather than giving feedbacks individually with one-toone and face-to-face.This finding is consistent with the one in Antoniou and James"s (2014) study on exploring both oral and written feedback used by primary school teachers.Their study findings showed that teachers" oral reactions to children"s effort and products were the most overt aspects of formative assessment.The written feedback of all teachers was too general and short, without providing explanations in relation to the strengths and weaknesses of the work done or how improvements could be made or maintained.According to Narciss and Huth (2004), feedback should provide the learner with suggestions for how to improve rather than correct answer.Also, praise can make pupils feel good but it does not help their learning unless it is explicit about what the pupil has done well (Swaffield, 2008).Moreover, majority of teacher participants were used to provide feedbacks to the whole class or by groups, rather than giving feedbacks individually with one-to-one and face-to-face.These findings in the present study may be due to teacher respondents lack teacher skills in providing feedbacks which are meaningful and advanced for student learning.Besides, the large number of students in the class limited the manner in which teachers provide feedbacks to students.Although the teachers realized providing feedback as important, they perceived offering feedbacks individually as a time-consuming process which could result in heavy burden; hence, the focus of attention tended to be the class, or student groups, rather than the individual student.
These findings are themselves of interest, but they also reveal the limitations of interviewing as a data collection procedure.What is evident from the data of all the interviews is that, when asked in an interview about their explicit understanding of key concepts, teachers may fail to produce a satisfactory answer.This may be due to any of a number of factors.If the question is asked at the beginning of an interview, before they have psychologically settled down, they may be flustered and confused, especiallyas in this casethey are asked to distinguish two very similar terms.In these circumstances, too, they may not have established a sound rapport with the interviewer, and thus be still working out the researcher"s agenda, leading to uncertainty about, and even suspicion of, the motivation of the latter; even the normal process of seeking and obtaining the participants" informed consent to be interviewed may not allay these feelingsand might in some circumstances enhance them.There is, too, the issue of the interpersonal factorsrespective age, gender, social or professional status, etc. -which may affect the development of a relationship between Copyright © 2018, IJAL, EISSN 2502-6747 the two.These potentially negative factors are also affected when the interview is conducted in the second language of the interviewee (and -as in this case -of the interviewer, the second author of this report).Apart from the matter of possible linguistic interferencepronunciation, syntax, etc.there are also sociopragmatic issues to take into account: the relative physical positioning and proximity of the interlocutors, how questions are framed, the extent and type of eyecontact, the importance attached to silence and hesitation, and a wealth of politeness factors.
Thus, the interviewer should not take first responses at face value.Tactfully, they need to probe beneath the surface of what the interviewees say at first to get a fuller picture.As can be seen from the later extracts from the interviews, it seems that most of the participants had an implicit understanding of the notion of formative assessment and, in some cases the actual term was explicitly used; this may indicate that they were at first confused.Moreover, they incorporated these tasks into their normal teaching practice, and used them to provide feedback to their students.Or, at least, they said that they did.
This raises the inevitable problem of interviews, questionnaires, narratives and other self-report procedures of collecting data: the truth value of what people say.Why should people tell a researcher what they really believe or think?And what is truth anyway?There are inevitably filters between the objective reality of what happened and its later recall and subsequent reporting.Time always lapses between an event and what people recall, and in the meantime other events, experiences and feelings may cloud an accurate recall.When the event is then reported, the interpersonal relationship between the individual and his/her interlocutor will also influence what is said; for example, one does not say exactly the same thing to a colleague as one does to one"s manager; different aspects are mentioned, different emphases are placed, etc.Thus, the report of any event is essentially constructed as a narrative to meet the purpose of the reporter and that assumed of the listener.

CONCLUSION AND IMPLICATIONS
Educational policies should take into account the existing knowledge and practices of those most directly concerned with its implementationthe teachers, as executive decision-makers in the classroom.Unfortunately, too often this is not done.One can think of curricula intended to introduce task-based learning in high schools (Hasim & Tunku Mohtar, 2013), instruction through the medium of English in universities (Barnard, 2014) or primary schools (Wong, Kumar & Barnard, 2009), or the intention to develop autonomy among university students (Barnard & Li, 2016).These are only some of many curricular policies which have failed to be effectively realised because the teachers were not consulted about the practicalities in the light of their experiential understanding.The question arises as to how the perceptions of teachers can most effectively be sought and obtained.
From what has been discussed above, self-report is potentially unreliable.This is most obviously the case with questionnaires, which can usually only scratch the surface of what the respondents think, partly because they are almost invariable completed in a very short time, and also because in many cases thoughts and words are put in the respondents" mindsby multichoice options or statements to which they are expected to (strongly) agree or disagree (strongly), or practices which they have to report along a frequency line from never to always.Thus, questionnaires may well be invalid in that they may not actually ascertain what the respondents think for themselves, and unreliable because the responses may not, for one reason or another, be truthful or honest.
As has been indicated above, the same may be true of interviews but many of the limitations can be overcome if they are conducted ethically, sensitively and at an appropriate place and time.Another problem with interviews is the time needed to conduct them, and this does not only mean the actual interview itself, but the time needed for the participants to be at the same place at the same time.The use of contemporary technology can reduce this time-factor through the use of email, or other forms of social networking such as (synchronous or asynchronous) oral or written messaging through Skype, blogs, wikis, etc.Such technology also permits eliciting information from more than one person.Thus, using focus groups of four or five interested parties, whether online or face-to-face, is more time-and cost-effective than the conventional form of one-to-one interviewing.Focus groups also have the advantage that they may well enable the participants to share knowledge, insights and experience, and possible also co-construct potential solutions to perceived problems.In this way, they may enhance the development of a community of professional practice within a schoolin addition to providing valuable information to policy-makers.
Finally, to obtain a clearer and fuller picture, selfreport procedures should be followed, wherever possible, by actual classroom observation; in this way convergences and divergences between what teachers believe and say they do and their actual practices can be made manifest.Given that there are always likely to be constraints to teachers hindering their attempts to put their beliefs into practice, it would be wise to discuss the lesson with the teacher to identify what some of these constraints might be, and to elicit the teacher"s rationalisation of decisions made in the classroom.
Evidently, to undertake such in-depth studies requires a considerable amount of researchers" time (and skills) as well as willingness and availability on the part of their participants.This points to the need to select appropriate contexts for case studies.If these are done systematically, the procedures and finding explained transparently, and the interpretations and implications justified by the findings, then they will Copyright © 2018, IJAL, EISSN 2502-6747 provide more valid and reliable evidence of teachers" cognition and practice than a wider-scale project which necessarily cannot provide detailed perspectives.
Therefore, it is suggested not only that policymakers should ascertain the beliefs and existing practices of teachers through empirical research before formulating a curricular policy, but should do so using appropriate procedures for doing so.
So what I did was from the group activity, whichever group that finishes early, so I congratulate them, say "awesome job", any kind of praises.I give them compliments, I do.I do compliment them for good work, for example, if the handwriting is too terrible, I cannot read the handwriting.I"ll say "you can do better than this", T5:T10: If they give me good answer for example, so I say, "Good job, thumbs up, very good."In writing, we"ll write "very good."I give them stars and sometimes, I will give them small presents to make sure they"ll to do better next time.