The Effect of Portfolio Assessment on EFL Learners ’ Reading Comprehension and Motivation

This study highlighted the humanistic-transformational perspectives via portfolio assessment which offers a conceptual framework for teaching and assessment. More specifically, it attempted toexplore the effectof portfolioassessmenton EFL learners’reading comprehension ability and motivation in the context of Iran. It adopted the quasi-experimental design comprising the pretest-treatment-posttest paradigm. To achieve the purpose, the researchers collected the triangulated data about the participants. Two classes were selected as the experimental and control groups from TabaranInstitute of Higher Education. They were 65 female university students majoring in translation. The only difference between the two groups was integrating portfolio into learning strategy-based instruction for the experimental group (portfolio-based instruction vs. non-portfolio instruction). At the post-testing stage, the both groups were retested through the reading comprehension test and the motivation questionnaire. A self-report assessment was also utilized to increase the credibility of the motivation test. The resultobtained from Mann Whitney U tests and t-testsrevealed that portfolio assessment as a constructivist strategy empowers participants’ reading comprehension and motivation.


Introduction
In the age of constructivism, the professional teachers need to employ alternative assessments in order to offer the learner's cognitive and affective feedback.However, in the context of Iran, the traditional testing culture is still prevalent in ELT classes.In fact, the over-reliance on the psychometric test undermines the ethical values and fairness.To mitigate the problems inherent in this regime, teachers can gain a wealth of information via alternative assessments.According to Brown (2004), the alternative assessments such as portfolio assessment might foster humanistic and constructivist atmosphere in language learning and teaching.Portfolio assessment (PA) as a tool links teaching and assessment in the educational arena.On the other hand, motivation is stipulated as an essential feature in language learning (Dornyei, 2001).As to PA and motivation in reading comprehension classes, Nowruzi and Nafisi (2010) maintain that PA is an effective tool in empowering the learners' psychological and cognitive ability.Reversely, Cambell and Mainak (2010) reveal that PA has a negative effect on the participants' attitude toward writing courses.In general, due to the controversial results and insufficient evidence particularly in reading comprehension classes which are of high significance in EFL classes, this study intends to explore if PA can have a significant role in promoting Iranian EFL language learners' motivation and reading comprehension ability.

Portfolio Assessment
The last few years have witnessed the paradigm shift in language teaching from the productandtransmission to the process and transformation of knowledge.The evaluation process has also undergone a changefrom a testing culture to an assessment culture (Gipps, 19994).While the former emphasizes the standardization of the test, the latter underlines what students are learning and what they can do with their knowledge (Brown, 2004).This innovative system of assessment is regarded as an alternative to the traditional testing.In fact, the alternative assessments emerging out of constructivism hold the view that in the complex and uncertain world, alternative assessments such as portfolio assessment (PA) provide diverse representations of learners' activities and strategies (Lynch, 2001).A number of definitions of portfolio have been adopted in the literature.For instance, to Yang (2003), a portfolio is a collection of students' activities documenting their effort and progress in their process of learning and their reflection on the materials.Brown and Hudson (1998) acknowledge the appealing face of portfolio assessment (PA) by stating that PA links teaching, learning, and assessment within the discipline.Indeed, a portfolio displays how each individual learner thinks, analyses, creates and interacts with teachers and peers linguistically, socially and emotionally (Grace, 1992).Therefore, PA is incorporated into teaching strategies (Porter & Cleland, 1995) which improves standards not measure them.A number of researchers point out that a portfolio is more than a product and it is a product-process based assessment which fosters learning ability (Altinay, Altinany, & Aytekin, 2008;Hamayan, 1995;Rolheiser, Bower, & Stevahn, 2000).It involves self-diagnosis, self-improvement, and themeta-cognitive processes of thinking (O'Malley & Pierce, 1996;Yang, 2003).From the critical perspective, Lynch (2001) states that PAnurtures fairnessand the balance of powerrelationships in theclassroom.Gottlieb (1995) discusses the two effective PA models for ESL as: Cradle model, and Moya and O'Mally's model.She describes the Cradle model as the process of Collection, Reflection, Assessing, Documenting, Linkages and Evaluating.She also explains Moya and O'Mally (1994) model byenumerating important features of a portfolio like comprehensiveness, systematic, informative, tailored and authentic.While the former model is student-centered, the latter stresses the teacher as a guiding factor in the development of portfolios.
As to assessing reading comprehension, Alderson (2000) suggests two techniques of summary and information transfer.Indeed, in transforming information the learners transfer what they have comprehended from the texts to a table, a chart and the other types of graphs.The other strategy proposed by Bos and Voughn (2002) is the "RAP" paraphrasing strategy for teaching reading comprehension.The learners are instructed to read a paragraph, ask themselves about the main idea and two details of the paragraph and putting them in their own words.The acronym for these stages is like rapping or talking to someone.In this regard, the semantic and syntactic components seem to be basicfor evaluation (McCarthy, Guss, & McNamara, 2009).

Motivation in Social Constructivism
There is also a considerable body of literature on motivation and its great significance in language learning.One of the most influential conceptualization of motivation is Gardner's dichotomy of integrative and instrumental orientation (Ellis, 2008).Nevertheless, the series of studies conducted by Dornyeiand his colleagues (2001, 2005, p. 5) challenge Gardner's integrative motivation.So, the researchers try to further define and conceptualize motivation in a foreign language.They disconfirm traditional motivation models developed in Western countries for second language learning.Many theories such as integrative orientation without adaption may not be relevant in all contexts and they may be inapplicable to foreign language situations.Gonzales (2010) argues that integrative orientation is far less important in FL settings.The contributors such as "teachers' style, competence, rapport, self-confidence, classroom atmosphere and group cohesion" play significant roles to motivation in the educational settings.Hence, motivation is an eclectic and multifaceted construct.Dornyei (1994) developed a socio-educational model of motivation particularly designed for a foreign language (William & Burden, 1997, p. 94).This model consists of different components influencing language learners: The language level, the learner level, and the situation.Accordingly, motivation, as Williams and Burden (1997) claim, is "very much context bound but it is amenable to change".
Alternatively, Deci and Ryan (2008), elaborating their earlier definition (dichotomy of intrinsic/extrinsic motivation), develop and analyzeself-determination theory of motivation.Woolfok, Winne, and Perry (2003) define self-determination as "the need to experience choice and control in what we do and how we do it" (p.374).In other words, what determines our actionis directed by the self rather than by external determinants (Pae, 2008).Deci and Ryan (2008) advocate that intrinsic and extrinsic motivation can be represented along a continuum relying on the rate of self-determination.Although the theory of self-determination has changed over the pastyears, the focus is on the role of the "dialectic between the person and environment in the satisfaction of the major psychological needs" (Vallerand & Pelletier, 2008, p. 257).Nakata's study (2006) on motivation and experience in foreign language learning indicates that educational experiences are relevant to ways in which such learners develop certain types of motivation.This also reflects the view that motivation is context-dependent, multifaceted, and dynamic; since it involves interaction between environment, learner's experience and his behavior.Adopting a social constructivist view of motivation, he demonstrates that intrinsic motivation, autonomy and goal are developed through social interaction.Ziahosseini and Salehi (2008), assessing the subjects' motivational orientation, conclude that despite the extrinsic motivational orientation, Iranian EFL learners are intrinsically motivated.

Research on Portfolio Assessment
Studies concerning the effectiveness of PA reveal that it promotes learning ability.Altinay et al. (2008) claim that a portfolio as a constructivist strategy enhances the learners' transferable skills so that the learners can use the skills such as reflection, critical thinking and assessment more professionally in the real life situation.Nowruzi and Nafisi (2010), examining the influence of PA on Iranian students' skills, highlight the positive effect of portfolio development of concept map on improving the learners' language ability.Furthermore, researchers have investigated the role of the PA in learners' motivation (Gambell & Mainak, 2010;Nowruzi & Nafisi, 2010;Crosby, 1997;Nezakatgoo, 2011).The findings are controversial.Nowruzi and Nafisi (2010) display that portfoliohas a positive effect on the learners' motivation.Reversely, Gambell and Mainak (2010), by employing attitude survey, interviews, and open-ended questions, illustrate that the subjects have a negative attitude towards PA in their writing courses.

The Statement of the Problem
Although recent research has consistently underlined the key role played by motivation in language learning, it is the main problem for the EFL students.Particularly, Iranian EFL learners are not highly motivated even though they need English in the educational arena and for telecommunication in the light of the emerging process of globalization.According to recent studies, motivation is an eclectic and a context-based construct (Dorneyei, 2001(Dorneyei, , 2005;;Gonzales, 2010;Birjandi & Hadidi, 2010) affected by geo-political and social factors.Moreover, a number of studies (e.g.Nezaktgoo, 2011;Porter & Cleland, 1995;Brown & Hudson, 1998) have been carried out to examine the effect of alternative assessments such as PA on the learners' motivation and language ability; however, rarely has it been applied to foreign language teaching, particularly reading comprehension (via the RAP strategy) which is the fundamental course in the academic settings in Iran.Consequently, we need to explore the effect of PA on learners' psychological and cognitive progress in an EFL learning environment.In this regard, two questions were raised for exploration: Q1: Does portfolio assessment affect EFL learners' reading comprehension?Q2: Does portfolio assessment affect EFL learners' motivation?
And to achieve the purpose, the following null hypotheses were formulated: H01: Portfolio assessment has no effect on EFL learners' reading comprehension.H02: Portfolio assessment has no effect on EFL learners' motivation.

Participants
To test the null hypothesis, 65 female university students among the total number of 82 students majoring in translation was selected from two classes to participate as the informants (The number of control group was 32 and the experimental group 33).They took the course of reading comprehension (III) at Tabaran Institute of Higher Education in Iran.The course was offered for second year students.The participants' age ranged between 18 and 24.Both groups (through the pretests) were almost homogenous regarding their motivation and reading comprehension ability at the outset of the study.Moreover, the two groups involved in this study were taught by the same teacher in order to provide the uniformity of instruction.The only difference between them was the instructional method: non-portfolio vs. portfolio-based instruction.

Instruments
To ascertain the present study enjoyed appropriateness, the researchers used the triangulation process to collect the data from multiple sources.The Michigan reading comprehension tests and standard motivation tests were employed as the pre-tests and the post-tests.The adapted questionnaire, originally developed by Birjandi and Hadidi (2010), was piloted and administered to the participants.It was designed to measure particularly the university students' motivation in Iran.In this regard, the motivation construct was operationalized in terms of multi-dimensional aspects of motivation towards foreign and specific courses.It was a five-point likert scale ranging from strong disagreement to strong agreement.The alpha coefficient reliability index of the motivation questionnaire was computed (0.87).Moreover, to enhance the credibility of the motivation construct, the researchers used a self-report assessment.The participants (both groups) were asked to report their probable changes of motivation towards the course and reading comprehension ability.The other instrument was the portfolio assessment (PA) which helped the researchers to assess the learners' progress during the course.

Instruction
The participants attended the 4-credit course of reading comprehension (III) for two sessions per week during one semester.The experimental group enjoyed the portfolio-based instruction besides the strategies and activities conducted in the control class.In the first session, the teacher explained the implementation of PA to the experimental group.The purpose, the components, the tasks of PA as well as the criteria for assessment were discussed.The students were trained how to record and collect what they have comprehended from each paragraph by using the RAP strategy (proposed by Bos & Vaughen, 2002) for portfolio implementation.More specifically, they were instructed to read a paragraph, to ask themselves about the main idea and details of each paragraph, and to put them in their own words into a reading strategy chart.So each lesson can be paraphrased by the RAP strategy.After strategy training and writing several models on the board (both the teacher and the students were involved), the students were guided to assess their own portfolios (self-assessment).In this sense, the teacher through one-to-one conferences with students evaluatedthe portfolios during the term.The evaluation was conducted in terms of certain criteria (according to Gottlieb's model, 1995).Finally, the portfolio was graded through the 4-point likert scale (from 1 to 4 assigned to each criterion).It is worth mentioning that the portfolio assessment is an alternative assessment which evaluates the students' progress.Therefore, in this study, PA is regarded as a tool for instruction and alternative assessments (Porter & Cleland, 1995).

Procedure
To assess the effectiveness of PA on the learners' motivation and reading comprehension (the course objective), the present study adopted the quasi-experimental design and the pretest-treatment-posttest paradigm.Two classes were selected as the experimental and the control groups (65 students in both groups).At the pretesting stage, the researchers administered the standard reading comprehension and the motivation tests to ascertain the homogeneity of the groups regarding the variables under investigation.At the instruction stage, the two groups attended the reading comprehension classes for one semester.The teacher, the textbook, the instruction were identical in the both groups except the portfolio-based instruction.In fact, portfolio assessment (PA) was implemented as an instrument to bridge the instruction and assessment.At the start of the course, the teacher (one of the researchers) explained the development and implementation of PA to the experimental group.Thus, the experimental group received the same activities in the control group as well as the portfolio-based instruction.After two sessions of training concerning the development and evaluation of portfolio, the teacher guided them to record their writings (through the RAP strategy) in a chart.Moreover, some students were asked to present their writings in the class and receive comments on them from their peers.The teacher evaluated and graded the students' generated portfoliosin terms of predetermined criteria.Furthermore, the teacher (through conferences and comments) helps the students diagnose their problems and improve their comprehension ability.So, self-assessment and peer-assessment can be fostered through the implementation of PA.Additionally, the portfolio-based instruction provides the processes of both feedback and feed forward.At the post-testing stage, the both groups were tested through the same tests of reading comprehension (the Michigan test) and the motivation test.Moreover, to increase the credibility of the motivation test, the researcher utilized a self-report assessment through which students were asked to report their motivation.
Afterwards, through the independent t-tests and the Mann-WhitneyU tests, the obtained data were compared and analyzed (the tables are illustrated in the results).

Results
At the pre-testing stage, the researchers administered the reading comprehension test in order to ascertain the homogeneity of the two groups concerning their reading comprehension ability.As depicted, the distributions of the two groups' scores are of normality assumption.Theskewness ratiofallsbetween -1.96 and +1.96 in the both groups.Also, the two groups enjoyed homogenous variances (F=3.735,P=0.58>0.05).Thus, an independent t-test was legitimized in order to compare the mean scores of the control and experimental groups.Table 2 displays the result of the Leven's test of homogeneity of variances and the t-test.As illustrated, the difference between the mean scores of the two groups is not significant (T=0.938,P=0.352>0.05).Since the p value was higher than 0.05, the two groups werehomogenous in terms of their reading ability prior to instruction.Concerning the participants' motivation, in order to be convinced of their homogeneity, theresearchers employed a motivation questionnaire to pretest the two groups before the treatment.The Mann-Whitney U test was usedto compare the experimental and the control groups.Table 3 shows the statistics.The findings manifested that the difference between the experimental and the control groups was not statistically significant.(U=557.000,p=0.815>0.05,two tailed).Thus, the two groups were homogenous concerning their motivation at the outset of study.
After instruction, the both groups were re-tested in terms of the variables under investigation.Asto the participants' reading comprehension ability, the scores in both the experimental and the control groups were normally distributed.Table 4 shows the skewness ratio of the two groups.Then, a t-test was run to measure the probable change between the two means of scores on the post-tests.Table 5 below displays result of the Leven's test of homogeneity of variances and the t-test.Table 5 reveals that there is no significant difference between the variances of the two groups (F=0.023,P=0.880>0.05).The independent t-test was run to compare the mean scores of the experimental and the control groups.As shown, the difference between the two groups was statistically significant (T=5.225,P=0.000<0.05).
Thepvalue was smaller than 0.05.Thus, the first nun hypothesis stating that portfolio assessment (PA) has no significant effect on EFL learners' reading comprehension was rejected.The result indicated that the experimental group outperformed the control group on the reading comprehension test.Accordingly, PA had a significant effect onthe subjects' reading comprehension ability.Figure 1 illustrates the comparison between the control and the experimental groups regarding their reading ability.Similarly, at the post-testing stage, the experimental and the control groups were also re-tested in terms of their motivation (through the same motivation questionnaire used in the pretests).The findings obtained from the Mann-Whitney Utest revealed a significant difference between the groups (u=403.500,P=0.049<0.05,two tailed).The participants in the experimental group enjoying portfolio-based instruction ranked higher in the motivation test than the control group.Table 6 displays the result.Hence, the second hypothesis was also rejected and it can be concluded that portfolio assessment (PA) had a significant effect on the learners' motivation.Figure 2 displaysthe comparison between the performance of the two groups on the motivation tests.As to the self-report assessment, the students stated that their motivation afterthe implementation of portfolio was enhanced.

Discussion
As mentioned before 65 female university students were selected in this study.They were randomly assigned to experimental (33 students) and control groups (32 students).In order to investigate the research hypotheses, the researchers utilized different proceduresfor collecting the data.At the pretesting stage, the homogeneity of the two groups were tested regarding their reading comprehension ability and their motivation.After instruction the same instruments were employed to test the variables under investigation.As to reading comprehension, the result obtained from the independent t-test showed that the experimental groups significantly outperformed the control group at 0.05 level of significance.Thus, PA has a significant effect on the participants' reading comprehension ability.Concerning the students' motivation in reading comprehension classes, the result of Mann-Whitney u test, comparing the performance of the two groups on the motivation questionnaire, revealed that the experimental group's motivation was importantly higher than the control group.The self-report instrument also illustrated the same result and the students reported that PA promoted their motivation toward reading comprehension classes.

Conclusion
Portfolio assessment (PA), in the present study, was found to have a significant effect on EFL learners' reading ability as well as their motivation.The participants in the experimental group performed better on the reading comprehension test when they received portfolio-basedinstruction.Additionally, they ranked higher on the motivation questionnaire.The result of the self-report assessmentincreased the credibilityof the motivation questionnaire.Therefore, the experimental group's motivation was promoted.Since at the outset of study, the two groups were homogenous, it can be concluded that PA has a positive effect on their reading comprehension and motivation.Thus, PA could be regarded as a tool (Vygotsky's terminology) in enhancing reading ability and motivation of the participants.The fact is that PAcould affect both psychological and cognitive ability.In this regard, motivation and reading comprehension were concerned as they are very important variables in academic settings.Additionally, motivation was evaluated in reading comprehension classes.In brief, the findings of this study corroborate Yang (2003), Brown and Hudson (1998); Porter and Cleland (1995) who claim that PA is influential in the EFL context.It is also in line with Crosby (1994, in Nezakatgoo, 2011), who highlights the significant role ofPA in promoting motivation.
PA fosters collaboration, learners' agency and autonomy that may contribute to enhancing motivation and language learning.The other merit of portfolio-based instruction in teaching reading comprehension is that students can reflect on their own artifacts.It can also improve their transferableskills such as reflection, critical thinking, cooperation and assessment (Altinay et al., 2008;Keshavarz, 2002;Tsagari, 2004).
As to the implications of the study, the literate assessment teachers can diagnose their own weaknesses and improve their own teaching strategies.They can also evaluate the students' styles and strategies to scaffold those who need assistance.Moreover, PA might facilitate the participation of students' voice, particularly the reflective and reserved learners.It can be aneffective tool in assessing and teaching bicultural and multicultural students.Indeed, PA might be as a viable alternativetool inboth teaching and assessment (Porter & Cleland, 1995).As an alternative to the testing culture, it mitigates the problems concerning normative, linguistic, and cultural biases inherent in traditional tests (Brown, 2004;Brown & Hudson, 1998); PA could also nurture social equity and fairness in EFL classes, so it is of significance in ethical assessment.The other implication is the RAP strategy (Bos & Vaughn, 2002) and using charts for the development of the students' portfolio in reading comprehension ability.
It is worth mentioning that one of the limitations of this study was the subject selection.Two intact classes (only female university students between 18 and 24) were selected.Among all skills, only reading comprehension and among all psychological variables, motivation were taken into account.In this regard, further research is suggested to focus on the fundamental questions: (a) Does PA, as an instructional method, result in enhancing the other skills and motivation?(b) Is PA an effective instrument for Dynamic Assessment?More explicitly, can PA be used effectively to measure the learners' potential ability?(c) To what extent can PA be correlated with the learners' transferable skills?
In sum, in the age of accountability, assessment literacy and teacher trainingare required to make teachers familiar with the new skill of portfolio development.To overcome the thorny issues on ELT, teachers are also suggested to take multi-perspectivism in language teaching and assessment.

Figure 1 .
Figure 1.Mean scores of the experimental and control groups on the reading pretests

Figure 3 .
Figure 3. Mean scores of the experimental and control groups on the motivation pre tests

Figure 4 .
Figure 4. Mean scores of the experimental and control groups on the motivation posttests

Table 1 .
Table 1 reveals descriptive statistics obtained from the control and experimental groups.Descriptive statistics on the reading comprehension pre-tests

Table 2 :
Independent t-test on the reading comprehension pretests

Table 3 .
Mann-Whitney u test on the motivation post tests

Table 4 .
Descriptive statistics on the reading comprehension post-test

Table 5 .
Independent t-test on the reading comprehension post-test

Table 6 .
Mann-Whitney u test on the motivation post tests