Constraints in Teacher Training for Computer Assisted Language Testing Implementation

Many ELT examinations have gone online in the last few years and a large number of educational institutions have also started considering the possibility of implementing their own tests. This paper deals with the training of a group of 24 ELT teachers in the Region of Valencia (Spain). In 2007, the Ministry of Education provided funds to determine whether it would be possible to implement an online University Entrance Examination (P.A.U.) in Spain at the national level. The project was to address three main areas: the technology, the students and the teachers. In relation to the teachers, the focus of this paper, the project was to investigate whether there were any changes in their routines, teaching methodology and attitudes towards assessment using technology. This brief study focuses on one of the last tasks in the preparation stage for implementation of the computerized exam, and it is intended to predict and observe the teachers’ reactions towards computer assisted language testing, the new test and the new test design. The findings shed light on the teachers’ internal changes and their changes in attitude throughout the process.


Introduction
Computer assisted language testing (CALT) has influenced the way in which teachers approach language testing (Chapelle & Douglas, 2006).Many ELT tests have gone online in the last few years and a large number of institutions have also started considering the possibility of implementing their own online tests.This paper deals with the training of a group of 24 ELT teachers in the Region of Valencia in Spain.In 2007, the Ministry of Education granted financial support to the CAMILLE group of the Polytechnic University of Valencia to determine whether it would be possible to implement an online University Entrance Examination (In Spanish, prueba de acceso universitario; henceforth, P.A.U.) in Spain at the national level.The project was address three main issues: the technology, the students and the teachers.In relation to the teachers, the focus of the present paper, the project was to investigate whether there were any changes in their routines, teaching methodology and attitudes towards assessment using technology.This brief study focuses on one of the last tasks in the preparation stage for implementation of the computer based exam, a teacher training course, and it is intended to observe and report the teachers' reactions towards CALT, the new test and the new test design.The findings shed light on their internal changes and their changes in attitude throughout the process.The paper will not address the benefits of training on their attitudes towards technology but, instead, offer a general overview of the main difficulties found in the training period.In order to determine them, the research team observed three main areas: problems related to test awareness, the priorities in the new test tasks, and the teachers' difficulties in managing ICT.

Background: the Place of the P.A.U. Examination in English Language Teaching
The P.A.U. is by far one of the most important high-stakes examinations for many students in Spain.About 175,000 students take it every year.Despite its importance, however, there seem to be few studies in relation to how teachers approach preparation of their students for the test (the so called washback effect) (Rea & Scott, 2007).'Washback' denotes the influence of an examination on the teaching process (Alderson & Wall, 1992;Messick, 1996;Wall, 2000;Cheng, 2004;Hausenberg, 2006).Nevertheless, a few research notes and descriptive papers do assert that teachers are mostly influenced by the test tasks and their own experience.In practice, those tasks that are not included in a test usually receive little if any attention in the classroom activity (Yin, 2010).Consequently, students lack preparation in such skills as speaking (which is not included on the exam), while their knowledge of language use can be higher than their actual communicative skills (Koeppen et al., 2008).The two most neglected skills are listening and speaking but the most susceptible to washback activities are grammar and translation (Wen-jun, 2010), so in a new test with a greater emphasis on speaking, the test would not be a major force in driving instruction (Muñoz & Alvarez, 2009;Shih, 2009).
Additionally, English teachers in Spain have not developed a special interest in the new information technologies, and they hardly use computers or language laboratories for instruction.Besides, although most students of translation, language education and philology in Spain (and possibly in the rest of Europe) will probably spend many years (if not to say their whole careers) in teaching --in which assessment is a crucial part of their job--, they receive almost no preparation for assessment in their university lessons (Stiggins & Conklin, 2002).Thus, teachers are not aware of the different factors involved in language testing (Cheng et al., 2008), and at the same time they need to be trained in this area (Baartman et al., 2007) as well as in the implications of high-stakes testing beyond their students' need to obtain the highest score possible (Luxia, 2007).Consequently, in this "post-methodology" era (Kumaravadivelu, 2002), many instructors tend to use a limited number of tasks which may go against the idea of communicative use of language in education as both a means and a target (Savignon, 2007), even when doing so may go against their own beliefs (Saif, 2006;Turner, 2006) and thus lead to negative washback.Teachers can find abundant literature in the market to improve and focus their teaching, but Spanish teachers seem to rely more on their own intuition than on the professional literature (García Laborda, 2001).In fact, there is dichotomy between what teachers think will help their students and what they actually do in class (Adir et al., 2006).
Teachers spend a great deal of their time preparing their students for three main types of tasks: filling in exercises related to language use, reading small texts (up to 500 words) and writing short essays (usually around 200-250 words).However, according to García Laborda (2007), computer-based examinations allow for the inclusion of a large number of different tasks including receptive and productive oral tasks that can be more realistic than on a pen and paper test (a condition suggested by Wiggins, 1998).With all of the above considerations in mind, a training course for P.A.U.teachers should include three main areas: preparation for language testing awareness, methodology for oral tasks, and technology use.These areas were also the main types of constraints that the researchers found in the current study.

Data Collection
Twenty-four teachers were observed during a teacher training course in order to gather the data for this study.This course dealt with teachers training for the pilot Spanish University Entrance Examination.Specifically, their reactions and ways of approaching the tasks were observed and annotated.Afterwards, a 10 item test was administered to them on the computer.Their qualitative responses were grouped and analyzed according to the following topics: test awareness, difficulties in methodology, and constraints in the use of technology.

Problems in Language Test Awareness
The traditional approach to the P.A.U.orientation for teachers in Spain, which is also currently used by the raters themselves (Amengual Pizarro, 2009), is descriptive.Teachers are occasionally given a 2-4 hour workshop at a university in which the test administrators simply explain the format of the test and the rating criteria.The criteria are then normally implemented by the teachers when they give examinations in the classroom.In the training course for this study, the teachers were asked to revise their assessment procedures and then use what they had learned in the training sessions to address some specific case studies.As a result, we observed the following situations among the 24 teachers: 1) Testing is not considered a fundamental part of the teaching process.The teachers thought that it was enough merely to use the same or similar exercises to those found in the students' workbooks or those included in the teacher's book to prepare a good test.
2) Assessment is hardly ever seen as a process.Instead, it is viewed as a set of punctual acts of students' providing evidence of their language knowledge and teachers' giving scores.Thus, the principles of measurement and impact on the daily teaching process are usually ignored.
3) The teachers limit the assessment process to exams and, at times, to subjective observations, so decision-making can be highly influenced by subjective feelings which may not be consistent with the students' real potential and/or their language knowledge (and use).
4) The teachers believe that assessment influences motivation but they ignore to the extent to which this is the case and how testing can be made to function as a valid instructional tool.

Difficulties in Methodology for Oral Tasks
In addition to having a limited view of assessment and examination, teachers of English in Spain also tend not to use English extensively in the classroom.In fact, in a recent survey among second year students at the Polytechnic University of Valencia, 30 second year students in the Travel & Tourism degree program responded to a questionnaire in which they were asked, among other things, about the language of instruction in their high school English courses; 19 of them (63%) responded that Spanish had been used most of the time while only 4 responded that most of their classroom interactions had been in English.Therefore, it can be concluded that the policies and methods used in the classroom will have to be revised in addition to the intention to introduce a computer-based examination.
Among our findings after the training sessions we first found that some of the teachers in the study had originally been French teachers and that they still had great limitations in English.These teachers had had to change to teaching English (and to learn it!)because French is no longer used as a second language in most high schools.Some of the other problems we came across were the following: 1) The teachers limit their focus to preparing students for the P.A.U. to the aims and scope of preparation for the P.A.U.Since the communicative skills of speaking and listening are not part of the test, the teachers feel that they should devote their time to activities more productive for the exam while ignoring aspects such as thinking and language use and knowledge, which are irrelevant to the exam.
2) The teachers have given up.Many of the teachers acknowledged that they would like to devote more time to learning activities related to oral development but that the importance of the test is such that the students must master the written skills.
3) The teachers complain about their circumstances.The teachers in the study tended to excuse their behavior because their colleagues in the primary schools do not teach speaking either.They claimed that, by the time their high schoolers reach the last year when they have them as students, it is just too late to teach them speaking.

Constraints in Technology Use
Although the training course for this study had been advertized with the name "Using Technology for ELT," we received some surprising reactions to it.First, some of the teachers who attended the training sessions reported that they had been "forced" by their own schools to attend them.Second, one of the teachers responded in the initial questionnaire that she "hated computers".However, we found two kinds of more extensive problems: basic use of technology on the one hand and attitudes towards technology on the other.The former was handled through intensive training during the course, but we noted that the difficulty does not reside in the use of certain computer software programs but in the teachers' attitudes towards implementing technology in their teaching, the latter problem.It was observed in the training sessions that the significant difficulty does not lie in how to teach but in what to teach.Nevertheless, when the teachers were properly instructed in the new technologies, content became the key matter in language instruction.For instance, after 12 hours of instruction in a computer testing platform, the teachers felt more confident and, at least in the training role plays, they paid more attention to what was to be taught than to how to teach their audience to use or perform a task on the given platform.

Implications for Teacher Training
Obviously, teacher training programs need to show the usefulness and applicability of what is learned in sessions.In this sense teachers must be helped to manage effectively the applications that will be used in their actual classes instead of being provided a general approach to ICT.If this is done, it will make the training period meaningful and worthwhile.The following issues should also be considered: 1) Teachers should be helped to understand the tools that they will have to use and their role in the test.This means that teachers must be shown that computers can be used to obtain evidence of areas that have not been measured to date (i.e. the oral skills).
2) Fears natural to people not familiar with digital technology must be eradicated.Thus, teachers need to be helped to understand their own limitations and to overcome them through team teaching, support from their colleagues, and cooperation with the most skillful students.
3) Teachers must be made aware that the ongoing effort to stay up-to-date on technological advances is worthwhile and highly accessible after an initial stage, which could be called a "discovery experience".4) Teachers must be informed of the existence of forums and sites on the Internet where they can share and learn from the other teachers' experiences.Examples are Dave's ESL Cafe, TEFL NET, Humanising Language teaching: http://www.hltmag.co.uk/ or www.isabelperez.com,but many more can also be found.

Conclusions
Overall, it should be pointed out that technology is definitely only one of the very many factors that have an incidence on teacher training in CALT.While trainers do need to make technology accessible to the teacher trainees, trainers also need to emphasize the contents and tasks that will be measured (for example, the oral skills that can be assessed through computer technology) (Pan, 2009) as well as how to do so (Inbar-Lourie & Donitsa-Schmidt, 2009).Opportunities for further development should also be promoted alongside the possibility of teachers' future collaboration.In the case of this study the participants kept in touch with each other after the course and worked together on certain school projects.In addition, they learned to use the technology beyond the P.A.U.test.In general, teacher training courses can be difficult to run but an awareness of participants' potential fears and problems can facilitate the trainer's role.In this sense, this paper is only a first approach to a complex matter which has received very limited attention in the professional literature to date.
In relation to methodology, teachers should be aware of times when washback is not absolutely necessary.Institutions should also ask their students about their needs for washback because test-oriented courses may be focused differently from others according to circumstances (some courses may not demand a strong emphasis on biasing instruction) (Qi, 2005;Scott, 2007;Gao, 2009).Minimally students have "different constructions of the themselves as test-takers, the test, and their performance in the test" and their accounts will include their "expectations and explanations for success or failure as well as credit or blame" (Huhta, 2006: 326).At the same time, it should be noted that washback requires strong research support based on baseline studies (Wall & Horák, 2007) to support not only school practices but also language policies as observed in current tests (Shohamy, 2007).

Note
We would like to express our gratitude to Spain's Ministry of Education for supporting and funding this study under the PAULEX project (HUM2007-66479-C02-01/FILO).We would also like to acknowledge joint funding from the Consejería de Educación of the Madrid Autonomous Region and the University of Alcala's R&D program for new groups in 2010.