Studying the Effects of an EFL Curriculum for Young Adults in Brazil

The ability to speak English offers young adults in Brazil opportunities to prepare for the global job market and promotes cross cultural understanding, which are national curriculum goals for public education. The study reported in this article documents the implementation of the Empregabilidade, Tecnologia e Inglescurriculum, which was designed to expand educational and economic opportunity for young adults attending public schoolsin Salvador, Brazil by providing them with basic communicative proficiency in spoken English. The results of a mixed method curriculum study demonstrate that the published curriculum was implemented differently than intended. Results of the study of the relationship between the implemented curriculum and participants’ oral language proficiency in English demonstrated that while participants in two cohorts did not reach target levels of proficiency, proficiency levels for students in the second year of the implementation were higher than those of students in the first year. Participants had positive attitudes toward learning English at the end of the implementation. Learners’ level of language classroom anxiety may have affected the level of language proficiency acquired as measured by the oral language proficiency assessment designed for the study. School and teacher variables under investigation did not have a significant impact on student outcomes, though there was evidence that students with a teacher implementing the curriculum a second time scored higher on the oral language proficiency assessment than students of teachers in their first year of implementation.


Introduction
As a result of Brazil's increasingly prominent role in the global economy, foreign languages have gained status in the school curriculum.In regions like Brazil's Northeast, the ability to speak Englishcan offer young adults access to the job market through the tourism industry and promote cross cultural understanding, an important goal for a nation seeking to increase its ties to the world outside its geographic boundaries.The Empregabilidade, Tecnologia e Inglescurriculum (Meyer, 2009), referred to in the remainder of this article as Employability, Technology, and English, or ETE, is an English as a Foreign Language (EFL) curriculum designed to be used by public high school EFL teachers to increase educational and economic opportunities for young adults living in poor communities in Salvador, Brazil by helping them develop basic communication skills in English with the support of computer technology.
In this article, the author presents the results of a study investigating: (1) how the published ETE curriculum wasimplemented, and (2) the impact of the implemented curriculum on the young adults who participated in two implementations of ETE in Salvador.The findings have important implications for EFL educators and curriculum developers seeking to provide learners with opportunities to develop basic communicative proficiency in English in countries like Brazil, and program evaluators or researchers interested in studying the relationship between curriculum implementation and its impact on learners.

Rationale for ETE Curriculum Development
Many young adults living in poor communities in Brazil have not been benefiting from the country's recent economic growth (Barrionuevo, 2010).The ETE curriculum (Meyer, 2009) was designed to promote employability opportunities and to ensure young adults developed positive attitudes toward learning a foreign language, goals that areconsistent with national educational goals in Brazil.
Educational policies and practices in Latin America are shaped by political, economic, and social conditions (O'Connor, 1991;Braslavsky, 2000), and in the decade following the ratification of Brazil's constitution in 1988, educational policy makers undertook a major reform effort to establishnational curricular goals thatwould prepare young people for the 21 st century labor market and guaranteedemocratization through their active engagement as citizens in Brazilian society (Ministério da Educação, 2000).This effort culminated in the development of the Parámetros Curiculares Nacionais (PCNs), whichdescribe specific competencies and skills for all disciplines.Competencies and skills for foreign languages in the PCNs for secondary educationinclude analyzing and interpreting the social and cultural meanings embedded in foreign language texts and using the language as a means of expression, information sharing and communication (Ministério da Educação, 2000).
Although the national foreign language standards reflect a commitment to developing learners' ability to use a foreign language for communicative purposes, foreign language programs in public schools have not achieved this in practice.The EFL teachers and teacher educators who participated in the ETE curriculum initiative pointed out thatcontinuous EFL exposure throughout elementary and high school rarely leads to communicative English language proficiency.Various explanations were offered for the limited success of young adults in attaining communicative proficiency in English by the end of high school including: poor instructional materials, large classes, limited English proficiency of EFL teachers, thelack of a coherent scope and sequence for EFL curriculum, and an emphasis on grammar and translation in most college entrance exams.
In Brazil, English is the preferred foreign language in educational programming (C.Rajagopalan & K. Rajagopalan, 2005) and is in high demand.When public schools do not provide adequate opportunities for students to develop communicative language proficiency in English, non-profit organizations invested in increasing the employability of young adults with limited economic resources assume an important role in creating such opportunities.In 2009, the non-profit organizationEnterJovem partnered with the Office of the Secretary of Education in the state of Bahia and USAID to finance and oversee the development and implementation of the ETE curriculum with the explicit purpose of enriching the lives of young adults through EFL instruction.The stated goals for the ETE curriculum were to: sensitize young people to the importance of English as a world language and as an instrument for empowering them to develop personally and professionally; stimulate positive attitudes towards learning English; help them develop specific learning strategies that accelerate learning a foreign language; and enable them to learn to communicate orally in English for basic functional purposes such as engaging in "small talk" and interacting with English speaking tourists.

Research Base for ETE Curriculum
To meet its stated goals, ETE curriculum content was organized thematically into four broad areas: developing an awareness of the importance and prevalence of English in the world (Theme 1); developing basic language learning strategies (Theme 2); exchanging personal information (Themes 3 through 10); and using English for communication in local settings frequented by tourists such as restaurants or stores (Themes 11-13).Content was addressed through activities that integrated language learning strategies, communicative oral and written language practice around specific language functions, and theme specific vocabulary and grammar development.Published curriculum materials included a teacher's guide, a text for students, and a CD for students with language samples.The curriculum also included activities requiring the use of computer software to enhance language learning.The approaches to language learning built into the curriculum weregrounded in research oncommunicative language teaching and learning (Griffins, 2011;Hadley, 2001;Savignon, 1983), and research on adolescent and adult learning (Bernaus & Gardner, 2008;Gorsuch, 2009;Lesgold & Welch-Ross, 2012).To support effective communicative language learning, the ETE curriculum included:(1) an emphasis on developing oral language proficiency that is purposeful and needed for real world situations; (2) practical applications of learning strategies that foster the learner's autonomy and independence; (3) integrated communicative, linguistic, and vocabulary building activities; and (4) exposure to English through technology.Because learner characteristics such as motivation, interests, and learning preferences have been shown to affect learner outcomes for adolescents and adults, the ETE curriculum reproduced situations in which English might be used in young people's life or job contexts.In addition, language tasks were designed to explicitly build on prior knowledge of the learners as a scaffold for the development of strategic competence and metalinguistic awareness, and included strategic uses of the native language as a support for exploring learning strategies.
In order to maximize the potential for the ETE curriculum goals to be met, professional development was provided by a Brazilian teacher educator who served as a staff developer.Professional development included an institute to familiarize teachers with the curriculum prior to the first implementation and weekly full day meetings and classroom assistance from the teacher educator while the curriculum was being implemented.

Method
A mixed method study was undertaken to investigatehow the published ETE curriculum was implementedand whether the implemented curriculum effectively developed learners' ability to speak Englishfor basic communicative purposes and stimulated positive attitudes toward learning English.

Participants & Sampling Procedures
Of 194 students between the ages of 16 and 21 who completed the ETE course over two implementations, 93 students were randomly selected from those who completed all assessments and consented to participate in the study.Cohort 1 included 49 students in the first year of the curriculum's implementation, and Cohort 2 included 44 students in the second year of implementation.The two cohorts were demographically similar and were students whose prior exposure to English had been in their public school classrooms.Any students who reported extensive exposure to English outside of school, e.g., participation in gaming online in English, were not included in the study.
The teachers who implemented the ETE curriculum were nine Brazilian EFL teachers released by the Office of the Secretary of Education from their regular teaching positions in public schools to implement the ETE curriculum.Six teachers participated in the first implementation and three were replaced by new teachers in the second implementation.Because of the emphasis on oral communication in the ETE curriculum, the project staff had requested the release of advanced English speaking teachers to participate in the implementation; however, the released teachers demonstrated English proficiency levels ranging from basic to advanced, which in fact, is more representative of EFL teachers in public schools in Brazil.
Over the two year implementation period, nine public schools in poor and working classroom communities in Salvador provided a computer classroom for ETE curriculum implementation, though there was variation in the condition of the computers and two schools did not have Internet access in the classrooms.

Research Design
In order to document the relationship between the intended ETE curriculum as described in the published curriculum materials (Meyer, 2009) and the curriculum as it was implemented, a mixed method design was adopted for the study.The mixed method design allowed for a careful documentation of how the ETE curriculum was implemented in addition to documenting the effects of its implementation on learners.
The choice of research designwas grounded in the work of curriculum scholars who recognize that an "intended" curriculum, e.g., a published curriculum like ETE, will not be implemented in the classroom with complete fidelity to its original design.Scholars argue that fidelityis unachievable because learners, teachers, and school ecology shape how the intended curriculum is put into practice (Clandinin & Connelly, 1992;Ferraço, 2011;Hlebowitsh, 2010;Macedo, 2011;Pinar, Reynolds, Slattery & Taubman, 1995).Teachers use their teaching knowledgeto select and modify content and develop their own conceptions of the curriculum in practice, and as result, there is the potential for every curriculum to be implemented in ways originally intended by its developers as well as ways that are unintended (Ben-Perez, 1990).
By using a mixed method design, the researcher was able to capture the differences between the intended ETE and the implemented ETE curriculum so that the findings regarding the impact of the curriculum on learners provided an authentic evaluation of the ETE curriculum that the learners actually experienced in classrooms as opposed to what the publish ETE curriculum intended for them to experience.

Data Collection
Mixed methodology involves both qualitative and quantitative data collection and analysis.Qualitative datacollection for the study focused on the curriculum implementation process and included documentation of classroom visits, observations of two professional development meetings, e-mail correspondence with staff, and teacher feedback on presentations of preliminary findings at the end of each implementation.
Quantitative data were collected on students' oral language performance and their attitudes and feelings about learning English.To assess the level of oral language proficiency acquired by learners, the Oral Language Proficiency Assessment (OLPA) was developed using the Language Proficiency Guidelines of the American Council on the Teaching of Foreign Languages ( 2001) and the ACTFL Integrated Performance Assessment (Glisan, Adiar-Hauck, Koda, Sandrock, and Swender, 2003).The OLPA included: (1) an interview protocol, which included a series of prompts developed by the researcher to undertake a conversation with a learner around the topics they were exposed to in the implemented curriculum, and (2) rubrics for the novice levels of speaking from the ACTFL guidelines.
In the ACTFL Guidelines, novice language proficiency is divided into three levels: low, mid, and high, and incorporate measures of comprehension, communication strategies, comprehensibility, language control, and the range of language functions in which the speaker is expected to participate.At the highest level, novice speakers should be able to: (1) successfullymanage a number of uncomplicated communicative tasks in informal/social situations; (2) talk about a few predictable topics such as sharing basic personal information, preferences and immediate needs; and (3) express themselves by relying heavily on learned phrases or recombinations of these and what they hear from the native speaker.Novice High speakers use mostly short and sometimes incompletesentences in the present tense and may speak hesitantly or inaccurately, but can be understood by an individual used to interacting with second language learners.
The OLPA was administered in the final two weeks of each implementation of the curriculum.For the assessment, each student spoke with the same native English speaker and all conversations were video-taped.Conversations lasted from 5 to 10 minutes.Once OLPAs were completed for a cohort, interviews were reviewed and assigned a holistic score of 1 (Novice Low), 2 (Novice Mid), or 3 (Novice High).
To assess the relationship between the implemented curriculum and learner affective variables, the Language Attitude and Motivation Survey (LAMS) was created using the Attitude/Motivation Test Battery developed by Gardner (1985) and grounded in research on the relationships between language classroom anxiety, attitudes and motivation, and achievement in learning a new language (Masgoret & Gardner, 2003).The ETE survey included 34 questions from the scales and relevant subtests of Gardner's original test (Note 1).
The LAMS was administered at the end of the implementation for Cohort 1, but for Cohort 2, the survey was administered at the beginning of the implementation.As a post-curriculum instrument for Cohort 1, the survey results provided information about the attitudes, motivation, and anxiety levels of learners at the end of implementation.As a pre-curriculum instrument for Cohort 2, the survey results provided information aboutpotential relationships between learners' predispositions toward language learning before being exposed to the ETE curriculum and the language proficiency levels they reached after experiencing it.

Procedures for Data Analysis
Qualitative and quantitative data were analysed separately.A content analysis was undertaken of the qualitative data to identify factors that emerged from the teachers' interpretations of the curriculum materials during implementation and influenced how the curriculum was experienced by the students.Comparative analyses of OLPA and LAMS datafrom students were conductedusing Pearson correlationand ANOVA.

Results
The results are presented in three parts.The first part describes how the implemented ETE curriculum compared to the intended ETE curriculum.In the second part, theresults of the effects of the implemented ETE curriculum in preparing students to use English for communication purposes and stimulating positive attitudes toward learning English are reported.In the third section, findings regarding particular learner, school, and teacher variables that may have influenced the outcomes of the implemented curriculum are presented.

Description of ETE Curriculum Implementation
The analysis of qualitative data on curriculum implementation demonstrated that the implementedETE curriculum differed from the intended one in content scope and in how curriculum materials were interpreted by teachers in practice.Differences were also found in how the curriculum was implemented during the first year and how it was implemented the second year.

Implementation for Cohort 1
In the 180 hours allocated for instructionfor Cohort 1, the first ten of the 13 themes in the published ETE curriculum were implemented.Implemented themes included: raising students' awareness about the presence of English in the world around them, developing their language learning strategies, and engaging them in activities to acquire the language necessary for communicating in English on a range of personal topics.Cohort 1 participants were not exposed to three themes at the end of published sequence, which were the themes dedicated to using English in local settings.
During curriculum implementation the first year, all six participating teachers reported focusing more instruction on oral language development and communication when implementing the ETE curriculum than when teaching in their traditional public school classrooms, where all reported having engaged in grammar instruction 100% of the time.However, communication-based instruction varied from classroom to classroom.Four of the teachers reported that their instruction had shifted to 50% communication-based instruction and 50% grammar-based instruction, and two reported emphasizing communication-based more than grammar-based instruction.Data from classroom visits confirmed this variation in practice.The staff developer noted that the directions for some tasks in parts of ETE curriculum materials were open to interpretation, and a teacher with a traditional grammar based theory of instruction could implement the activity in a way that offered limited studen-to-student communication, for example, filling in a chart with the right words without having to "negotiate" with a partner.
In addition, time allocated toinstruction on individual themes reflected the influence of teachers' priorities.The first two themes, which focused on developing an awareness of English in the surrounding environment and studying language learning strateiges, were allocated more instructional time than the later themes.Because these two themeswere designed to tap students' prior knowledge and to encourage students to make connections between knowledge of their native language and English, many of the activities in the early themes incorporated the use of the native language, unlike later themes that focused more on English communication and practice.
In the final two weeks of the first implementation, as part of their professional development, the teachers were invited to attend the OLPA interviews with their students so that they could observe how their students performed when engaged in a conversation with a native English speaker.Once levels were assigned by the researcher, the teachers, researcher, and staff developer met together to review video samples of students who received high novice and mid novice ratings.This was intended to enhance teachers' understanding the communicative language expectations established in the published ETE curriculum.

Implementation for Cohort 2
The second implementation began later than originally scheduled due to a change in the Secretary of Education.This delay and a one week teacher strike led to a reduction in instructional hours in the second implementation from 180 to 120 hours.As a result, Cohort 2 students were exposed to the first nine themes of the curriculum.
During the second implementation, three teachers returned to teach a second cohort, and three new teachers began teaching the ETE curriculum for the first time.Changes were made in the professional development to support all of the teachers in interpreting the curriculum through a strong communicative lens.Weekly professional development sessions focused heavily on implementing communicative activities to encourage oral language practice, and there was an observed increase in the number of communicative activities implemented in the classrooms.In addition, more time was allocated for the implementation of themes related to exchanging personal information.As a result of these changes, Cohort 2 students were exposed to more English language practice than students in Cohort 1 despite the reduction in total hours of exposure to the curriculum.

The Effects of the Implemented ETE Curriculum on Learners
In this section, the impact of the implemented curriculum on learners is reported.The results include findings regarding: (1) the oral language proficiency acquired by students as a result of the implemented curriculum, and (2) learners' attitudes toward and motivation to study English following the implementation of the curriculum.

Oral Language Proficiency
One of the goals of the ETE curriculum was for students to develop basic communicate proficiency in English.In both cohorts 1 and 2, fewer than 15% of the students reached a Novice High level on the OLPA.However, when the results for all three novice levels were compared, it became evident that students in Cohort 2 developed more communicative language proficiency than students in Cohort 1.As can be seen in Table 1, the mean score increased from 1.57 for Cohort 1 to 1.84 for Cohort 2. In Cohort 1, more than half of the students (57%) did not move beyond a novice low level, meaning they had no real functional ability in the language.However, in Cohort 2, there was a sizeable increase in the number of students reaching the Novice Mid level of oral proficiency (57%) despite a reduction in total hours of exposure to the curriculum during the second implementation.
Consistent with descriptors from the ACTFL rubric, the Novice Mid level speakers had developed the ability to engage in very basic exchanges around some language functions to which they have been exposed in the ETE curriculum, but in contrast to Novice High level speakers, they communicated with difficulty, using one or two word responses or stock answers to questions, and pausing frequently when speaking.

Student Attitudes toward Learning English
A second goal of the ETE curriculum was for students to develop positive attitudes toward learning English.
Results from Cohort 1 data suggest that students completed the curriculum with positive attitudes toward learning Englishand were highly motivated to learn the language.As can be seen in Table 3, mean scores were high in all subtests on the survey, suggesting that participants held positive attitudes about learning another language and toward English speaking people and culture, and demonstrated a strong motivation to learn English after completing the curriculum.

Learner, Teacher & School Variables Affecting the Outcomes of Curriculum Implementation
In the curriculum studies literature, scholars have argued that learners, teachers, and school ecology shape how an intended curriculum is put into practice.If such variables influenced how the ETE curriculum was implemented, then they could influence the outcomes.For this reason, the current study included an investigation of four variables that could potentially affect the outcomes: learner classroom anxiety, school setting, teachers' English language proficiency and the amount of experience teachers had with ETE implementation.

Learner Classroom Anxiety
For Cohort 1, results of an analysis of the relationship between subtest scores on the LAMS and the OPLA scores suggest that success on the OLPA might be linked to a learner's level of anxiety in learning another language..008 As can be seen in Table 4, a comparison of results from the LAMS subtests and OLPA scores for Cohort 1, there was a positive correlation between language classroom anxiety and OLPA scores (.408, p ≤ .01),meaning that the lower the level of anxiety, the higher the score on the OLPA.
Cohort 2 participants were administered theLAMS before experiencing the ETE curriculum to provide insight intostudents'pre-existing attitudes, motivation and anxiety before being exposed to the ETE curriculum.No statistically significant relationships were found between the results of the LAMSsubtests and OLPA scores.However, there was a small positive trend toward significance between OLPA scores and the subtest on language classroom anxiety (p = .091),suggesting a possible relationship between a student's level of anxiety at the beginning of the course and OLPA performance at the end.

School and Teacher Variables
Results from a comparison of school sites where the curriculum was implemented and participants' OLPA scores showed no significant differences in the English proficiency levels acquired by the participants at different sites.
Results of a comparison of OLPA scores and the English language proficiency level of the teachers also showed no significant relationship between the two variables.However, an analysis of the relationship between the OLPA scores and the number of years of experience that teachers had with the ETE curriculum (and the professional development that accompanied it) suggested a possible relationship.Though the difference was not statistically significant, the mean score on the Oral Language Proficiency Assessment (OLPA) for students with teachers implementing the curriculum a second time was higher (X=1.84)than the mean score of students with teachers implementing the curriculum for the first time (X=1.62).This suggests that students with a teacher in her second year of implementing the curriculum were more successful at communicating in English with the interviewer during the OLPA.

Discussion & Implications
The findings of the present study demonstrate that there are important differences between a published curriculum and how it is implemented in classrooms.To investigate the impact of an EFL curriculumdesigned to developbasiccommunication skills in English and to stimulate positive attitudes and motivation toward learning English on learners, researchers must evaluate the implemented curriculum, as was done in the current study.
Results revealed the important role that teachers play in shaping curriculum in practice.They also illustrated how well designed assessments can capture the impact of an implemented curriculum and the variables that might influence the outcomes including teachers' experience with the curriculum and learner language anxiety.Below, these findings are discussed in more detail.

The Important Role of the Teacher in Curriculum Implementation
Findings from the study of the ETE curriculum implementation demonstrated that the transition from grammar/translation based to communication based teaching was a challenge for EFL teachers whose previous experience was limited to grammar based instruction even though they had positive views of communicative language learning, a finding consistent with studies of EFL teachers in other countries (Karavas-Doukas, 1996;Saqui, 2004).Although the ETE curriculum materials were designed for communication practice, teachers interpreted them through personal frames of reference that were grounded in a grammar/translation orientation.The findings suggest that changing this orientation requires professional support and time.
Cohort 2 participants moved closer to meeting the curriculum goals than Cohort 1 participants, and students had better chances of reaching a higher level of oral language proficiency by having a teacher who was implementing the ETE curriculum a second time.An emphasis on communicative pedagogy in professional development combined with repeated exposure to the curriculum materials through opportunities to enact them more than once canimprove students' chances of meeting the intended goals of a communicative curriculum.The teachers who participated in both implementations had the added benefit of witnessing their students' engage with a native speaker during the OLPA interviews at the end of the first implementationand to view videos of students achieving different levels of proficiency.These kinds of professional opportunities can clarify for teachers standards-based expectations, which may differ from what they envision based ontheir own theories of practice.
Current efforts to improve foreign language education in Brazil are focusing on the selection of foreign language textbook series that purport to meet the competencies and skills identified in the PCNs and guarantee that students in public schools are able to comprehend the foreign language and produce it orally and in writing (Fundo Nacional de Desenvolvimento da Educação, 2011).Findings from the current study would suggest that simply providing teachers and students with access tocurriculum materials will be insufficient to ensure that students meet the competencies focusing on language use unless EFL teachers receivemeaningful professional development on how to implement the communicative elements of thenew materials.Resources that address common concerns of EFL teachers such as providing learners with language focused feedback (Hunter, 2012) and effective approaches to balancing the use of the native language and the target language (Sampson, 2012) canbe used todesignmeaningful professional development programs to accompany EFL curriculum implementation.

Understanding Relationships between Learner Attributes and Language Learning
Although theimplemented ETE curriculum met its goal of stimulating positive attitudes and motivation toward learning English, results for Subtest 3 onthe LAMS for Cohort 1 revealed a relationship between language classroom anxiety and performance on the OLPA; students with higher levels of anxiety did not perform as well on the OLPA.The operational definition of language classroom anxiety and the questions on the LAMS measuring the construct were adopted from Gardner's original study (1985), and measure a participant'sdegree of comfort or discomfort while participating in language class activities.The results from the current study supports research connecting language classroom anxiety to EFL student performance on oral language assessments (Atasheneh & Izadi, 2012;Hewitt & Stephenson, 2012).
Research has demonstrated that foreign language teachers can reduce student anxiety through interventions that help them build confidence (Atasheneh & Izadi, 2012), and that exposing learners to the kinds of communicative tasks and the expectations to be met in those tasks throughout curriculum implementation can lower the anxiety levels of learners in testing situations (Gorsuch, 2009).This would suggest that curriculum implementation incorporate repeated opportunities for learners to engage in the kinds of communicative tasks reflected in assessments used to measure learner outcomes.During the implementation of the ETE curriculum, teachers were not provided opportunities to alignthe daily assessment activities built into the curriculum with the tasks in the Oral Language Proficiency Interview.This alignment might have produced better results for those learners who showed high levels of classroom language anxiety, and should serve as a starting point for future research.

Using Valid Assessments of Communicative Language Proficiency
The OLPA designed for the present study provided a valid heuristic for evaluating the effects of a communication based curriculum by providing evidence of participants' ability to interact with an English speaker in a communicative context.Because it was a measure of oral language proficiency, it provided valid data on how the implemented ETE curriculum fell short of meeting its target for having learners develop a basic proficiency in English.
Developing meaningful assessments of oral language proficiency is critical to evaluating the outcomes of an EFL curriculum withoral language communication as its goal.The challenge in countries like Brazil is to develop a cost effective assessment of communicative language use when thousands of learners must be assessed.As technology becomes more sophisticated and more accessible, the possibility of assessing learners' communicative performanceusing computer-based testing procedureswill becomes easier to realize.Models already exist; the web-based directory of over 200 assessments for languages other than English developed by the Center for Applied Linguistics include examples of computer based assessments (Center for Applied Linguistics, undated), and additional resources for test development are available through organizations like the Council of Europe (2001).

Measuring Learner Attitudes & Motivation
Findings from the current study support research that points to a strong positive relationship between language acquisition and the attitudes and motivation that learners bring to the language learning context (Gardner, 1985;Masgoret & Gardner, 2003).The Language Attitude/Motivation Survey was used for different purposes for Cohorts 1 and 2. Capturing the knowledge and dispositions of students prior to their participation in curriculum implementation can assistcurriculum evaluators and researchers in exploring the interface between a school curriculum and the curriculum teachers and students bring with them to the learning context.It would be interesting to use the LAMS as both a pre-and post-assessment to determine if there was any change in participants' affective characteristics during curriculum implementation.

Studying Curriculum Implementation
The study reported here illustrates the distance between an intended EFL curriculum and the one that is implemented.Curriculum study designsmust capture the complex and multi-layered effects of curriculum implementation across different learning contexts.Studying the implementation process can assist EFL curriculum developers in designing more realistic goals and expectations, and they can inform decisions regarding the instructional time required for meeting curriculum goals.For example, findings from the current study suggest that more curriculum exposure was required for learners to meet the goals of the intended ETE curriculum given the many variables that affected the implementation process.Research on short, intensive foreign language education programssuggest a minimum intensive exposure of 240 hours of instruction (Hadley, 2001), significantly more time than was allocated for the implementation of the ETE curriculum.
In writing about curriculum studies in the Brazilian context, Macedo (2011) argues that "the curriculum that matters is that practiced in schools, in clear contrast to the formal documents and policies" (p.138).For practice to be more closely aligned to policy, it will benefit policy makers and teacher educators to better understand the variables that shape curriculum implementation and the effects that these variables can have on learner outcomes.

Table 1 .
Mean scores on OLPA & percentages for proficiency levels by Cohort

Table 2
results shown in Table2suggestthe differences between cohort scores might be significant (p =.065), with Cohort 2 students scoring better than Cohort 1 students on the OLPA.

Table 3 .
Mean scores for OLPA and LAMS subtests from Cohort 1