Evaluation of Modular EFL Educational Program ( Audio-Visual Materials Translation & Translation of Deeds & Documents )

Modular EFL Educational Program has managed to offer specialized language education in two specific fields: Audio-visual Materials Translation and Translation of Deeds and Documents. However, no explicit empirical studies can be traced on both internal and external validity measures as well as the extent of compatibility of both courses with the standards and criteria of scientific educational program. In a bid to address these issues, this study was conducted to evaluate the program from five fundamental criteria including: Admission Requirements, Program Content, Program Resources, Program Instruction/Evaluation Methods, and Graduation/Employment Requirements. Methodologically, the study is based on the requirements of both qualitative and quantitative research paradigms. To this end, a sample of teachers enjoying at least five years of offering both courses attempted a 22-item Likert-scaled questionnaire accommodating subcategories of the five macro criteria followed by open-ended written protocol commenting spaces for qualitative data. The findings revealed controversies over the all the macro-criteria and compatibility of the program with these well-established standards; suggesting exercise of comprehensive revisits and modifications in all aspects of the program as a whole.


Introduce the Problem
One of the most crucial prerequisites of conducting an effective educational program, particularly foreign language learning program, is having a clearly defined curriculum with specific predetermined objectives.Taba (1962) cited by Tunc (2010) notes that "A curriculum usually contains a statement of aims and of specific objectives; it indicates some selection and organization of content; it either implies or manifests certain patterns of learning and teaching.... Finally, it includes a program of evaluation of the outcomes." (p. 11).This highlights the significance of having codified standards for a curriculum in specific and an educational program in general.The predetermined international standards act as an impeccable criterion for the curriculum upon which it can be evaluated.From this point of view, carrying out evaluation studies in order to comprehend strengths and weaknesses of educational programs, applying modifications and determining the degree of consistency between standard educational systems and other systems in other contexts seem to be an essential mission of every successful educational program.Richards (n.d.) claims that a language curriculum should be viewed as "…a network of interacting systems involving teachers, learners, materials, schools, administrators and curriculum planners, and choices at one level affect other elements in the system."(p.13).Similarly, Kucuk (2010) states that a language teaching program is "a series of foreign language courses teaching the language through some kind of methodology so as to fulfill aim/aims such as communication or passing a proficiency exam." (p. 53) Language education in Iran is run by both public and private sectors.Although the Ministry of Education has a full supervision over the public institutions, private sector is mainly managed on financial basis and is viewed as a business opportunity for its stakeholders.Therefore, it is probable that such organizations lack necessary qualifications for a successful educational program.There are some dimensions which address the quality management in educational systems in general and language educational systems in particular.Accordingly, Richards (n.d.) refers to some aspects that need to be constantly examined in order to maintain the quality of teaching and learning in language educational programs aligned with standards.They are "… design of the curriculum, the quality of instructional materials, the role of tests, provisions for teacher training, and the kinds of administrative support provided by schools and educational institutions."(Richards,n.d.,p. 1).Referring to the large volume of research done in reaching methods and techniques, he also criticizes this excessive attention paid to detailed aspects of language teaching and suggests a shift towards the context of teaching and more specifically to the quality teaching.

Program Evaluation
There has been a notable attention to evaluation and its uses in recent years.Evaluation research varies depending on who carries out the study, which methods are used and how findings are implemented.Murphy (2000) defines evaluation as a way to determine the degree in which a program is effective in terms of its objectives.At the same time, it gives support to stakeholders in decision making for program improvement through careful analysis of information gathered.Kiely (2009, p.99) asserts that " evaluation has evolved from focused studies of teaching methods inspired by language learning theories to a curriculum management enterprise with a focus on quality assurance and enhancement".

Language Program Evaluation
Balint (2009) asserts that there are abundant publications in general educational program evaluations compared with language specific evaluation studies.However, there has recently been a substantial amount of books and in journal articles on language program evaluation whose dates imply the fact that the Language Program Evaluation is a recent issue to a certain extent.He follows the language program evaluation development overtime by providing a table taken from Kiely & Rea-Dickins (2005).

Purpose of Study
The main purpose of the study was to evaluate the effectiveness of two courses of Modular English Language Educational Program, including Audio-visual Materials Translation and Translation of Deeds and Documents Courses as well as the extent of their compatibility with standards of a conventional educational program, from the perspectives of EFL instructors involved in offering them for years.

Method
This research study was conducted with a randomly selected participation of 30 teachers.To collect the required data, a teachers' questionnaire as a modified version of a curriculum evaluation questionnaire for the Private Training Institution, Division of Institutional and Industrial Education, Department of Education in Newfoundland, Canada was employed.In order to ease the data analysis procedure, open-ended questions had accompanied by a Likert Scale (Strongly Agree, Agree, Neutral, Disagree, and Strongly Disagree) an open ended.

Results
Table 2 displays the answers given by the teachers to the relevant items in order to probe whether Modular English Language Educational Program is aligned with standards of an effective educational program.41.8 of the respondents (36.1 + 5.7= 41.8) believe that the Modular English Language Educational Program is aligned with standards of an effective educational program.On the other hand, 41 percent (30.5+10.5)believe that the Modular English Language Educational Program is not aligned with standards of an effective educational program.About 17 percent of the respondents have expressed no idea about the issue under study.Pie Chart 1 displays the above mentioned percentages.
Figure 1.Percentages modular English language educational program An analysis of chi-square was run to probe whether the differences observed in Table 2 are systematic or random.
The results of the analysis of chi-square (x 2 (4) = 204.90,P = .000< .05)indicates that the differences observed in Table 2 are meaningful, i.e. the differences are not obtained by chance.Based on these results, the first null-hypothesis was rejected; indicating that the differences between those who hold positive and negative stances are not statistically significant..000a. 0 cells (0.0%) have expected frequencies less than 5.The minimum expected cell frequency is 121.8.
Given the nature of the dimensions of the first research question, six sub-questions addressing the respective dimensions of the program were statistically tested in the form of respective null hypotheses.

Minor Research Question 1
Are the teachers consistent in their description of Modular English Language Educational Program?
As displayed in Table 4 majority of the respondents, i.e. 41.67 percent of the teachers disagree with the descriptions provided on Modular English Language Educational.On the other hand 36.66 percent agree with the provided descriptions.21.67 percent are neutral.An analysis of chi-square is run to probe whether the differences observed in Table 4 are systematic or random.
The results of the analysis of chi-square (x 2 (4) = 20.83,P = .000< .05)indicates that the differences observed in .000a. 0 cells (0.0%) have expected frequencies less than 5.The minimum expected cell frequency is 12.

Do Modular English Language Educational Program admission requirements meet standards of an effective educational program?
Based on the information displayed in Table 6 it can be concluded that majority of the respondents, i.e. 22.88 percent, believe that the Modular English Language Educational Program admission requirements do not meet standards of an effective educational program.On the other hand 34.52 percent of the respondents hold the opposite view.22.62 percent are neutral.Pie chart 3 displays the percentages as appeared in Table 5.
Figure 3. Percentages modular English language admission requirements An analysis of chi-square was run to probe the second Minor research questions as the alignment of Modular English Language Admission Requirements with the standards of an effective educational program.The chi-square observed value of 15.52 (P = .004< .05)indicates that the frequencies and percentages observed in Table 6 show significant differences between the respondents' views.Thus, it can be concluded that the data fail to reject the second minor null-hypothesis addressing the Program admission requirements; the admission requirements not aligned with the present standards of an effective educational program.Pie chart 4 displays the percentages as appeared in Table 7.  8 show significant differences between the respondents' views.Thus it can be concluded that the third minor null-hypothesis as Modular English Language Educational content does not meet standards of an effective educational program is rejected.Majority of the respondents believe that the educational content of the Modular English Language are aligned with the present standards of an effective educational program..000a. 0 cells (0.0%) have expected frequencies less than 5.The minimum expected cell frequency is 41.2.

Do Modular English Language Educational Program Resources meet standards of an effective educational program?
Based on the information displayed in Table 10 it can be concluded that majority of the respondents, i.e. 51.39 percent, believe that the Modular English Language Educational resources do not meet standards of an effective educational program.On the other hand 33.33 percent of the respondents hold the opposite view.15.28 percent are neutral.Pie chart 5 displays the percentages as appeared in Table 10.

Figure 5. Percentages Modular English Language Educational Resources
An analysis of chi-square is run to probe the fourth Minor research questions as the alignment of Modular English Language education resources with the standards of an effective educational program.The chi-square observed value of 72.11 (P = .000< .05)indicates that the frequencies and percentages observed in Table 10 show significant differences between the respondents' views.Thus, the data fail to reject the fourth minor null-hypothesis..000a. 0 cells (0.0%) have expected frequencies less than 5.The minimum expected cell frequency is 28.8.

Minor Research Question 5
Does Modular English Language Educational Program Instruction and Evaluation Method require some revisions to meet standards of an effective educational program?
The frequencies and percentages of the respondents' views on the Program Instruction and Evaluation Method of Modular English Language Education Program are displayed in Table 12.Based on the information displayed in Table 12 it can be concluded that majority of the respondents, i.e. 59.30 percent, believe that the Modular English Language Educational Program requires some revisions suggested by teachers to meet Instruction and Evaluation Method meets the standards of an effective educational program.On the other hand 19.77 percent of the respondents hold the opposite view.They believe that the Program Instruction and Evaluation Method of Modular English Language Education do not require some revisions to meet the standards of an effective educational program.20.93 percent are neutral.Pie chart 6 displays the percentages as appeared in Table 7.  12 show significant differences between the respondents' views.Thus, it can be concluded that the fifth minor null-hypothesis is rejected..000a. 0 cells (0.0%) have expected frequencies less than 5.The minimum expected cell frequency is 21.5.

Minor Research Question 6
Do Modular English Language Educational Program Graduation Requirements meet standards of an effective educational program?
Based on the information displayed in Table 14 it can be concluded that majority of the respondents, i.e. 55.17 percent, believe that the Modular English Language Graduation Requirements do not meet standards of an effective educational program.On the other hand 31.03percent of the respondents hold the opposite view.13.79 percent are neutral.Pie chart 7 displays the percentages as appeared in Table 14.
Figure 7. Percentages Modular English Language Graduation Requirements An analysis of chi-square is run to probe the sixth Minor research questions as the alignment of Modular English Language Graduation Requirements with the standards of an effective educational program.The chi-square observed value of 4.51 (P = .211> .05)indicates that the frequencies and percentages observed in Table 14 do not show significant differences between the respondents' views.Thus, the data fail to reject the sixth minor null-hypothesis.Majority of the respondents believe that the Program Graduation Requirements of the Modular English Language are not aligned with the present standards of an effective educational program..211a. 0 cells (0.0%) have expected frequencies less than 5.The minimum expected cell frequency is 7.3.

Discussion
According to the table one, majority of the teachers have expressed their agreement towards the alignment of the Modular English Language Educational Program with standards of an effective educational program meaningfully.Based on these results, it can be concluded that the first null-hypothesis as Modular English Language Educational Program is not aligned with standards of an effective educational program is rejected.

Program Description
The program description seems to be sound theoretically; however, as it doesn't consider students' background knowledge and the time allocated is not sufficient, in practice, the outputs are not satisfactory.

Admission Requirements
The majority of teachers claimed that no entrance exam is held and students are selected on the basis of their high school Grade Point Average (GPA) which is normally below the average.Moreover, it should be noted that most of the students attending Modular English Language Education Program, are those who haven't been able to enter a state-run or Islamic Azad university.Therefore the University of Applied Science has been their final opportunity.

Program Content
They mostly found the sequencing of training, the pre-requisite courses, the time or number of credits, the objectives of every individual course, and the balance between theory and practice to be properly addressed.However, it should be noticed that there exist a slight difference between those who agree and those who do not.This is to say that the points mentioned by those who have expressed dissatisfaction about the program content shouldn't be overlooked.First of all, they indicated that the sequencing of courses is not orderly designed and there is lack of cohesion.Also, some difficult and specialized courses are offered in the first modules.Second, some other teachers pointed out that according to the university's curriculum each module focuses on a specific theme; therefore, including a massive amount of specialized courses in one module will bombard students with so many field-specific terminologies which will be soon forgotten in the next module.

Program Resources
Although, the educational branches vary in their educational facilities and equipments, the teachers mostly argued that most centers suffer from necessary requirements.Moreover, majority of the teachers agreed that the textbooks listed are not adequate for program delivery.They also noted that textbook selection is done subjectively by each department or based on the teacher's preference and in some cases the students are provided with ready-made pamphlets.Some other claimed that textbook selection should be done by resourceful teachers who are aware of learners' and situation needs.Many teachers stated that textbooks are rich in variety but poor in quality.

Program Instruction/Evaluation Methods
The teachers suggested regular and strong supervision on the classroom content should be exercised, teachers should to involve students in teaching process and classes should be student-centered.They also recommended that classes be held in workshops and the students be encouraged to analyze the syntactic structures of both Farsi and English so that they can be more aware of the similarities and differences between two languages.The teachers also observed so many shortcomings in students' evaluation methods.They also pointed out that there are rarely sound criteria for the students' acceptance and the students are judged by some non-academic criteria concerning their personal lives such as: marital status, working conditions and having children.They further, recommended a constant realistic assessment of the student's performance and project-oriented measurement alongside testing and suggested that teachers need to be stricter in applying the acceptance requirements.

Graduation/Employment Requirements
According to the results, majority of the teachers believed that the prerequisites for successful completion of the program are not sufficient and the graduation requirements do not meet the present standards of an effective educational program.Some claimed that most of the students do not have enough motivation; therefore, they merely think about passing the courses and getting the certificate.The students do not possess practical knowledge of what had been taught theoretically, thus, soon after they've graduated or even immediately after a course final exam, they forget what they had learnt.So, criteria of graduation should include both quantitative and qualitative measures.
Regarding the employment requirements, the teachers listed certain types of prospective occupational positions for the graduates as follows: 1. Working as translators or interpreters 2. Teaching opportunities 3. Job opportunities in film production studios However, the teachers believed that many of the graduates are not competent enough and minority of them can be proficient translators.Therefore, successful completion of the program doesn't necessarily mean that they are able to do the job.
Overall, majority of the teachers believed that the Modular English Language Education Program needs some modifications regarding its objectives.According to the teachers, the whole system is planned a bit unsystematically and financial concerns seem to play a much significant role than academic issues in program planning.They further mentioned some other shortcomings for the program.First of all, the course is sometimes a big challenge for the students who have serious problems in English.Besides, heterogeneous classes produced boredom and resentment for both teachers and studious students.Therefore, teachers suggest that the students be admitted on the basis of their language proficiency.Second, it's wise to evaluate the candidates' motivation and objectives in choosing the course and the university.Third, some teachers claimed that the materials should play the leading role in the classroom, however, in some cases the materials are overlooked by some other teachers.Fourth, it was claimed that there are rarely any sound justifications for presenting Translation of Film and cassette Course, as the centers which offer the course do not mainly have adequate educational facilities.Even majority of centers are not equipped with a decent language laboratory.Also, there are not enough suitable texts and the teachers are not expert in the spontaneous translation and interpreting.After all, according to the teachers, it can be concluded that drastic measures are needed to re-evaluate, modify and change the curriculum in terms of admission, planning, implementation, sequencing and graduation requirements.

Figure 2 .
Figure 2. Percentages modular English language educational program description

Figure 4 .
Figure 4. Percentages modular English language education content

Figure 6 .
Figure 6.Percentages Modular English Language Program Instruction and Evaluation Method Therefore, he identifies four factors including institutional factors, teacher factors, teaching factors and Learning factors, which play an important role in the success of language teaching program.
 Teaching factors: teaching model and principles; and evaluating teaching  Learning factors: understanding of the course; views of learning; learning styles; motivation; and support

Table 2 .
Frequencies and Percentages Modular English Language Educational Program

Table 3 .
Analysis of Chi-Square

Table 4 .
Frequencies and Percentages Modular English Language Educational Description Pie Chart 2 displays the above mentioned percentages.

Table 4
are meaningful, i.e. the differences are not obtained by chance.Based on these results it can be concluded that the first minor null-hypothesis addressing the extent of consistency of the course description is also rejected.Majority of the teachers believe that the provided descriptions do not hold true for Modular English Language Educational Program.It means that what is described as an area of a course is different from what actually exists.

Table 5 .
Analysis of Chi-Square Description of Modular English Language Educational Program

Table 6 .
Frequencies and Percentages Modular English Language Admission Requirements

Table 7 .
Analysis of Chi-Square Description of Modular English Language Admission Requirements Does Modular English Language Educational Program Content meet standards of an effective educational program?The frequencies and percentages of the respondents' views on the quality of the education content of Modular English Language Education Program are displayed in Table8.Based on the information displayed in Table8it can be concluded that majority of the respondents, i.e. 46.60 percent, believe that the Modular English Language Educational Program content meets the standards of an effective educational program.On the other hand 39.81 percent of the respondents hold the opposite view.They believe that the education content of Modular English Language Education does not meet the standards of an effective educational program.13.59 percent are neutral.

Table 7 .
Frequencies and Percentages Modular English Language Education Content

Table 9 .
Analysis of Chi-Square Description of Modular English Language Education Content

Table 10 .
Frequencies and Percentages Modular English Language Program Resources

Table 11 .
Analysis of Chi-Square Description of Modular English Language Educational Resources

Table 12 .
Frequencies and Percentages Modular English Language Program Instruction and Evaluation Method

Table 13 .
Analysis of Chi-Square Description of Modular English Language Program Instruction and Evaluation Method

Table 14 .
Frequencies and Percentages Modular English Language Graduation Requirements

Table 15 .
Analysis of Chi-Square Description of Modular English Language Graduation Requirements