Students’ Perceptions of the Effectiveness of the English Program Taught at the College of Technological Studies in Kuwait

This research aims at examining the Students’ perceptions of the effectiveness of the English program implemented in the College of Technological Studies in Kuwait. A sample of 242 students was randomly selected from the population. The students were asked to respond to a 21-statement questionnaire designed to obtain information about how they perceive their competencies in English after they have successfully completed the requirements of the English program. The statements of the questionnaire were distributed in such a way so that information can be obtained about five areas that have bearings on the attainment of the goals of the program and the students’ improvement in the four skills. Overall, the participants viewed the English program as effective in improving their English competencies. However, they also pointed to some inadequacies which seem to be related to the program’s implementation. In order for these inadequacies to be effectively redressed, the researchers recommend that further studies be conducted on other aspects of the programs, like the teaching methods and tools used in implementing it and feedback from instructors.


Background
This study aimed at evaluating the English language program taught at the College of Technological Studies (CTS), the Public Authority for Applied Education and Training (PAAET) in Kuwait from the perspectives of students who successfully completed it.In doing so, we hope to be able to contemplate on the likelihood of student' attainment of the goals of the program, and hence we will be able to gain valuable assessments of the impact of the program and, indirectly, its ability to fulfill the goals it is set for.
Participants in the study are students who are seeking a diploma degree in one of the majors in engineering from CTS and have completed successfully the compulsory English courses the English program offers.Upon joining CTS, students sit for the English placement test as part of the requirement for admission to of one the majors in engineering.According to the results, students are placed in the appropriate level of English.A passing grade in the placement test (60% or above) means that students take two English courses: Eng 101, which is designed to enhance the English skills they have learned previously in school, and Eng 171, an ESP course designed to improve their proficiency in the English skills needed in their specialization.A failing grade (less than 60%), however, means that students have to take a remedial course (Eng 99) in addition to the other two courses.This study is designed to explore how students perceive their improvement in the English skills upon successfully completing the courses that the program offers.As such, their perceptions can be one way of providing commentary on the effectiveness of the program and the attainment of its goals.

Purpose of the Study and Research Questions
A corollary premise that has long been acknowledged in educational research and practice is that improvement cannot be achieved without evaluation.Evaluation is the corner stone of any educational practice.In distinguishing evaluation from research, Stufflebean (1981) says: "The purpose of evaluation is to improve, not to prove."The statement to improve means that a judgment must be made with regard to what constitutes worth or value.In other words, improvement is directly linked to evaluation and the term evaluation is associated with making judgment about the value or the effectiveness of a certain action.
The main purpose of the study is to evaluate the effectiveness of the English program in the College of Technological Studies in Kuwait.More specifically, the students' perceptions of their improvement in English proficiency and the effectiveness of the program are examined upon completing all the courses offered by the program.The researchers' ultimate aim is to contribute to the improving the program and enhancing its performance.
The research poses two main and interchangeable questions: 1) How effective is the English program in improving the students' competencies in the English language skills?
2) What are the strengths and weaknesses of the program?

Significance of the Study
It is in this context that the present study can make significant contribution.The study attempts an evaluation of the English program taught in the College of Technological Studies, which disseminates English Language Skills to students joining this college.It is hoped that this study will provide policy makers and educators involved with this program at all levels with pertinent information that will help them evaluate this program and make informed decisions about it in particular and decisions with regard to the problem of the deficiency in English language that college students in Kuwait suffer from.The significance of the study arises from the fact that it is the first study of its kind that attempts an evaluation of the English program at the Public Authority for Applied Education and Training.Another significant aspect of this study is that it will contribute to the scant body of literature on evaluations of English programs in Kuwait.As such, the results of the study may be of assistance to other universities in understanding the deficiencies in their programs.

Literature Review on Evaluation of Educational Programs
Evaluation can be carried out for different reasons in all aspects of life.When it comes to education, it can be said that the main goal of evaluation is to get feedback about student and teacher performance, the curriculum and other aspects of the classroom interactions.Likewise, the goals might include the determination of strengths and weaknesses of certain educational activities in a given program.
As Worthen and Sanders (1998) point out, there is not a unanimous agreement on the definition of evaluation.Some educators associate evaluation with measurement; however, others describe it as the measurement of the degree to which the stated objectives have been achieved.Furthermore, some educators think of evaluation as basically a scientific investigation, while others contend that it is the act of gathering data and providing information to enable decision-makers to improve the performance of a program.Despite this lack of unanimity about evaluation, Murphy (2000) describes evaluation as a way to identify the extent to which a program attains its objectives.In this sense, it helps decision-makersin improving program by analyzing the data gathered.Kiely (2009, p. 99) asserts that "evaluation has emerged from studies on teaching methods which were stimulated by theories on language learning with a focus on quality assurance and enhancement."Talmage (1982) points out that evaluation is the processgiving judgments to recognize the value and merit of a given program.Moreover, "evaluations can differ on many dimensions, among them design (experimental, quasi-experimental, regression discontinuity) intent (advocacy versus objective assessment), philosophical underpinnings (quantitative versus qualitative), and others" (Frechtling, 2007, p. 104).Cronbach (1991) differentiates three types of decisions that need evaluation: 1) Course improvement: determining the adequacy of the teaching materials and methods and the need for change when required.
2) Decisions about individuals: recognizing the needs of the studentin planning his/her instruction, and making the student aware of his own performance and weaknesses.
3) Administrative regulation: gauging the quality of the school system, the individual teachers, etc.
According to Ralph Tyler (1991), evaluation is a process essential to curriculum development.Lynch (1996) differentiates between evaluation and assessment based on scope and purpose.He argues that evaluation can rely on assessment tools such as tests and many other instruments such as interviews.According to him evaluation is "…the systematic attempt to gather information in order to make judgments or decisions."(p. 2) The goal of evaluation is to identify the degree to which the program has attained achieved its stated objectives.Evaluation consists in the determination of strengths and deficiencies of a given program, a step that entails re-planning and implementation (Gredler, 1996).In the same way, Worthen and Sanders (1998) statethat evaluation is the identification of the quality, competence or merit of a program or curriculum.There are different methods carried out for evaluation during this process.These methods consist in setting standards for determining quality.Moreover, there is the method of gathering pertinent information.Finally, the standards are used to determine quality.In light of these definitions, we can say that program evaluation is a systematic investigation set to offer decision makers information that help improve the performance of a program.This investigation can be expressed in the form question like 'How does the program work?', 'Does the program produce unintended side effects?' (Cronbach, 1980, p. 87).As such, program evaluation focuses on the investigation of one or more of five program areas.a) the need for the program b) the design of the program c) the program implementation and service delivery d) the program impact or outcomes and e) program efficiency (cost effectiveness) (Payne, 1994, p. 15).Mackay (1994) points out that in the domain of foreign language teaching, program evaluation consists in different processes, ranging from academic to non-academic investigations.That is to say, evaluation may capitalize on a variety of areas of a language program, like curriculum design, classroom processes, the teachers and students.
Evaluation is integral to the educational process.In this capacity, it becomes a challenging mission.Kelly (1999) describes program evaluation as the attempt to measure the effectiveness of any educational program.As Lynch (1996) states, the two dominant objectives of program evaluation are assessing its effectiveness and its quality when compared to similar programs.Not only does program evaluation offer essential feedback and information to insiders regarding the enhancement of the current program but also provides accountability to outsiders.The purpose is to determine whether the program designed and implemented can produce the learning outcomes.In this sense, evaluation can capitalize on the progress and deficiencies of the program curriculum prior to implementation and the effectiveness of its implementation (Ornstein & Hunkins, 1998).Therefore, a systematic and ongoing assessment of a program is essential for its improvement, which might lead to the need for curriculum evaluation.Scriven (1991) connected evaluation with the time in which evaluation is conducted and, accordingly, came up with the concepts of formative evaluation and summative evaluation.Formative evaluation is conducted while the program is being designed.This kind of evaluation helps the designers in improving and amending the program.The purpose of formative evaluation is to validate or ensure that the goals of the instruction are being achieved and to improve the instruction if necessary (Weston, Mc Alpine, & Bordonaro, 1995).Therefore, it is obvious that formative evaluation offers information required for instant changes.Students' learning activities can be reconsidered rechanneled and the instructional activities of a curriculum can be revised (Tunstall & Gipps, 1996).In this sense, it focuses on both students' improvement and growth (Pryor & Torrance, 1996).In short, formative evaluation is carried out during the implementation of a program to offer decision and policy makers' useful data towards its enhancement.
However, summative evaluation is carried out after the implementation of a program to provide judgments about the value and merit of a program.For instance, after a curriculum is thoroughly developed, a summative evaluation might be required to identify the effectiveness of the curriculum package (Worthen & Sanders, 1998).Summative Evaluation is the ultimate objective of an educational activity.That is, summative evaluation provides the information on which decision making can rely.It offers data on the effectiveness of a program.Summative evaluation generally uses numeric scores or letter grades to assess learner achievement.This study conducts a summative evaluation of the English program in CTS since the program has been going on for fifteen years now, and the researchers are concerned with finding out whether the learners have learned what they are supposed to learn.That is, the study seeks to find out whether the outcomes of the programs relevant to the language proficiency have been attained after a long time of implementation.
Another study relevant to the present research is that of Zucker and Shields (1995, p. 1) who point out that evaluative initiatives or reforms assumes that students' attainment of high standards of learning requires three important elements: 1) significant improvement in classroom teaching, 2) the use of more challenging curricula and material, and 3) the regular assessment student learning integrated and aligned with instruction.This indicates that evaluation is as important as instructional curricula and their implementation.That is, evaluation is very crucial in indicating whether the outcomes are being achieved or not.It can be used, as Webb (1997, p. 4) says, to "formulate policy, monitor policy effects, enforce compliance with policies, demonstrate accountability, make comparisons, monitor progress toward goals, and/or make judgments about the effectiveness of particular programs" Indeed, evaluators, as Fitzpatrick (1998) maintains, have a very important advisory role: they help identify goals and develop strategies for accomplishing these goals.On the other hand, program developers, as Gamse and Millsap (2002, p. 2) remark, are documenting the effects or impacts of their proposed intervention strategies or reforms.

Population and Sample
The population of the study consisted of 600 students (both males and females) who are joining different major in the field of engineering at the College of Technological Studies.At the time of the study, the students were enrolling in ESP English 170 in the second semester of the academic year 2013/2014.A sample of 242 students was randomly selected from the population.The students of the sample should have completed Eng 101 (the prerequisite for the ESP Course) and are now enrolling in ESP Eng 170.The students were asked to respond to a 21-statement questionnaire which was designed to obtain information about how they perceive their competencies in English after they have successfully completed its requirements.

Instrument
A questionnaire of 21 statements was developed to obtain information about students' perceptions of the effectiveness of the English program taught at CTS.The items of the questionnaire were distributed in such a way so that information can be obtained about five areas that have bearings on the attainment of the goals of the program and the students' improvement if the four skills.These areas are: 1) Statements 1 through 5: Students' perceptions of their improvement in the reading skill 2) Statements 6 through 8: students' perceptions of their improvement in grammar 3) Statements 9 through 12: students' perceptions of their improvement in the writing skills.

Procedure
The revised version of the questionnaire was piloted on 60 students to check comprehension and clarity of the items.As a result, some modifications were made for more clarity and precession.The final questionnaire was distributed towards the end of the first semester, a week before the final examination period to ensure that students have completed the courses of the program.To further ensure that the students understand the statements of the questionnaire, and, therefore, would respond as accurately as possible, the questionnaire was translated into Arabic, their mother tongue.Students were given the choice to respond to the Arabic version or English version.

Placement Test
The English program in PAAET seeks to carry out the objectives and general philosophy of the overall curriculum, which is geared towards improving the English language skills and, hence, enhancing their performance in their future jobs.Upon joining CTS, students are required to take the English Placement Test.According to the results, students are classified as either Fail (if they earn a score of less than 60%) or Pass (if they earn a score of 60% or more).Students who fail the test are required to take a remedial course (Eng 99) and pass it before they venture into Eng 101 and ESP Eng 170.Those who pass take Eng 101 and ESP Eng 170.Eng 101 is a perquisite for ESP Eng 170, which means students cannot take Eng 170 before they successfully complete Eng 101.While Eng 101 is a general intermediate-level course intended to enhance the students' proficiency in the skills, ESP Eng 170 is an upper intermediate course with technical content and aims at equipping students with the English skills they need in the engineering majors.As we can see, the stated objectives of the English program address all the four skills.The study focuses on the extent to which these objectives have been realized from the students' points of view.

Results
In the analysis of the results, a scale of (disagree, don't know, agree) was used.During count of response occurrences, 8 cases were excluded as the respondents did not answer the questions.Of the 250 respondents, the total number of valid responses was 242, from which the frequency and percentage were calculated.

Reading Skill
An overview of the response to the statements targeting the reading skill shows that the participants believed that their reading proficiency in English was improved after they completed the program.As Table 1 shows, the participants responded positively to the first statement: 81.81% of the respondents believed that the program helped them read and comprehend various types of texts, as opposed to 11.57% who disagreed with the statement and only 6.61% expressed no idea.The same holds true of the third, fourth and fifth statements: 83.05% of respondents agreed that the English courses helped them strategies of increasing their vocabulary knowledge (as opposed to 8.67% who disagreed and 8.26% who expressed no idea); 74.79% of the respondents agreed that the English courses helped read and comprehend scientific texts (as opposed to 16.11% who disagreed and 9.09% expressed no opinion); and 65.58% of the respondents agreed that the English courses helped read between the lines (while 11.98% disagreed and 22.72 expressed no opinion).The second statement (After I finished the English courses, I was able to read and interpret charts and tables), however, shows the respondents' dissatisfaction with the ability of the English program to improve their skill of reading charts and tables.Only % 49.58 agreed with the statement, in contrast to 22.7% who disagreed, and 27.68% who expressed no idea.

Grammar
While the majority of the participant (81.81%) agreed with the first statement (The English courses offered comprehensive revision of the English grammar I learned in school), the responses to the second and third statements did not even halve the total number (42.56% and 45.86% respectively).The results regarding grammar proficiency indicate that while there is focus on revising the grammatical rules, when it comes to practice (writing grammatically correct sentences and analyzing structures) students believe that the courses did not offer much help.

Writing Skill
In contrast to the students' perceptions of the improvement in the reading skill, their perceptions of writing skill were not as favorable.That is, of the literacy skills, participants seem to believe that the English courses tend to focus on reading more than writing.Participants were not sure if they were really able to write well organized and coherent paragraphs: 49.17% agreed as opposed to 27.27% who disagreed and 23.55% who expressed no idea.The same holds true of writing reports of technical nature related to their engineering majors: only 45.86% of the participants agreed with the statement; 24.79% disagreed; and 29.33% remained neutral.As for the first and third statements, participants seem to be more confident that the courses helped them learn the skills of writing (62.57%) and punctuation marks and writing mechanics (59.55%).Again, like their response to the proficiency in grammar, participants seem to say here that the courses focused on the first and second cognitive levels of learning objectives (invoking Bloom's Taxonomy), but they didn't move up to a higher level like application of the skill.Since writing technical reports pertinent to the major is one of the learning objectives of the ESP course, participants seem to believe that this particular objective was not attained.

Oral Skills
As for the oral skills, participants expressed satisfaction with their improvement in communicating and listening in English as the results of the first statement indicate (62.81% agreed as opposed to only 17.17% who disagreed and 19.42% who suspended their views), the second statement (64.05% agreed with the statement; 14.05% disagreed; and 21.90% expressed no idea), and the fourth statement (72.31% agreed; 13.22% disagreed; and 14.46% expressed no idea).However, only 39.26% of the participants believed that the English courses improved their presentation skills, as opposed to a relatively close percentage (33.06%)who disagreed with the statement.This indicates that the English courses (particularly the ESP course) fall short of the objective related to improving the students' presentation skills.

Content of the Program
The majority of the participants agreed with the statements that the methods in teaching the courses were effective; the English courses improved their proficiency in English; and the courses enhances their skills in English.As the percentages scored on the first three statements indicate, to a great extent, the general goals of the programs were attained.The participants seem to agree that the courses offered by the program have improved their proficiency in English and enhanced their previous knowledge.However, in the last two statements participants point to a shortcoming in the implementation of the program; namely the focus on the reading skill more than any other skills (66.12% agreed with the statement, as opposed to 17.35% who disagreed and 16.52% who expressed no view) and the lack of focus on the oral skills (63.22% agreed; 24.79% disagreed; and 11.29% expressed no opinion).

Discussion
This study solicits students' perceptions of the effectiveness of the EFL program taught in the College of Technological Studies in Kuwait.It poses the following two questions about the program by way of examining the Students' perceptions: 1) How effective is the English program taught at CTS in improving the students' performance in the English language skills?
2) What are the strengths and weaknesses of the program?
To answer the first the question, we also by default highlight the points of strength that the program possesses.
Participants seem to think that overall the English program is very effective as it helped them improve their competencies of the English skills.Of the four skills, reading came to the forefront as respondents believed that their proficiency in the reading skill was enhanced to a great extent.Even when it comes to grammar, most of the students believed that the program provided them with an effective revision of the grammatical rules they learned previously.As for the writing skill, students' perceptions were positive on the acquisition of the skill and on the revision of the punctuation marks and the other writing mechanics, though they pointed out to some deficiencies with regard to learning the actual process of writing.With regard to the oral skills, the responses to the statements were overall positive as they indicated that their oral proficiency was improved after they completed the courses of the program.The effectiveness of the program in enhancing the respondents' skills was also confirmed by the responses to the last part on the content of the program: 71.49% of the participants agreed that the teaching methods in delivering the content of the program were effective.
On the other hand, the participants pointed out to some deficiencies that the program might be suffering from, which, if redressed, the program's performance can be enhanced and hence better results may be yielded.First, though the students' perceptions on the reading skill were generally positive, they were not as positive when they responded to the statement on learning the skill of reading and interpreting charts and Another weakness the results of the study point to is related to the balance, or rather lack of it, in designing the instructional experience for each skill.As we have seen, students believe that the English Program helped them improve their reading skill more than any other skill.This result, combined with the results for the other skills which were not as favorable, seems to suggest that in the implementation of the program there is a tendency to focus on the reading skill at the expense of the other skills.This is attested to by the students' response to the statement (The English courses focused on the reading skills more than any other skills) where 66.12% responded positively.
Of the three statements on grammar, the number of participants who agreed with the second and third did not even halve the total number: only 42.56% said the English courses helped them write meaningful and grammatically correct sentences; and only 45.86% said that the courses helped them analyze the structure of the English sentence.It seems that the learning experience for the grammar skill does not move from level of the recall and comprehension of information to higher levels of application and analysis (to refer to Bloom's cognitive levels of thinking).Again, this is an issue that solicits further research on the implementation of the program and the design of instructional experience for each skill.
The same holds true of the writing skill.The students' responses suggest that the English courses did not help them beyond revising writing mechanics and learning skills of writing.However, when it comes to writing well-organized and coherent paragraphs and reports pertinent to the engineering majors, the participants did not think that they made progress.Again, we can only speculate about the reasons behind this weakness.Like the case with grammar, it seems the instructional experience for writing does not take the students to a higher level of application.Needless to say, writing is a process which requires that students get involved in the actual experience of composition, not just learn about the skills.This means that in order for students to master such a difficult skill, they need to experience the processes of drafting, revising and editing in order for them to discover what language can do.
Finally, what has been said is also applicable to the presentation skills as the responses to the statements on the oral skills suggest that the goal for the presentation skills has not been attained.Only 39.26% agreed that the courses helped in advancing their presentation skill in front of an audience.It would be interesting to investigate this issue in a future research, in which the instructional experience designed the presentation skill is thoroughly examined.

Conclusion and Recommendations
This study aimed at examining the Students' perceptions of the effectiveness of the English program implemented in the College of Technological Studies in Kuwait.A sample of 242 students was randomly selected from the population.The students were asked to respond to a 21-statement questionnaire designed to obtain information about how they perceive their competencies in English after they have successfully completed the requirements of the English program.The statements of the questionnaire were distributed in such a way so that information can be obtained about five areas that have bearings on the attainment of the goals of the program and the students' improvement if the four skills.Overall, the participants viewed the English program as effective in improving their English competencies.However, they also pointed to inadequacies which seem to be related to the implementation of the program.One such inadequacy is the lack of balance in designing the instructional experience for each skill.Another is that the learning experience for some skills does not take the students to higher cognitive levels, like application, analysis and synthesis.A third inadequacy is that some of the stated goals of the ESP course have not been attained, like reading charts and tables and writing reports of technical nature.The main limitation of this study is that it relies on only students' perceptions of the effectiveness of the English program and how they perceive their own competencies As such one of the basic recommendations the researchers have come up with is that further studies be conducted on other aspects of the programs, like the teaching methods and tools used in implementing it, feedback from instructors, direct observations and exit examinations upon successfully finishing the requirement of the program.These studies are bound to complete the picture about the program's effectiveness, and certainly, would be a major step towards enhancing its performance.
One of the implications that seems to manifest itself in the results and discussion is that for any educational program to yield fruits, it must incorporate the process of assessment and evaluation as a part of its implementation.We, therefore, recommend that steps be taken right away to develop a coherent continuous system of evaluation that is in alignment with the objectives of the English program.This is necessary in order to provide very much needed feedback to all who are involved in the teaching process so that the program can be evaluated and improved.Information technology can be utilized to create databases and baseline information on students, teachers, courses and testing practices, so that evaluation processes and research projects can be easily conducted.

4)
Statements 13 through 16: Students perceptions of their improvement in the oral/aural skills 5) Statements 17 through 21: Students perceptions of the content of the English program.
Upon the completion of the English courses, students should master the following objectives:  Demonstrate comprehension of a range of printed and/or electronic materials  Maximize silent and oral reading fluency  Employ a variety of comprehension strategies  Develop vocabulary-building strategies  Respond critically to a wide range of printed and electronic materials  Use rules of grammar  Write for different purposes  Use sustained speech  Demonstrate control of grammar  Respond to listening situations  Follow telephone conversations  Speak understandably  Use goal-specific language  Analyze societal norms  Work cooperatively with diverse populations

Table 1 .
Students' response to the statements about the reading skill

Table 2 .
Students' response to the statements about grammar proficiency

Table 3 .
Students' response to the statements about the writing skill

Table 4 .
Students' response to the statements about the oral skills

Table 5 .
Students' response to the statements on the content of the program table (only 49.58% agreed).This seems to indicate that this goal, which is one of the stated goals of the ESP course, has not been attained according to students.It seems that instructors of ESP Eng 170 don't give due emphasis to this goal, or the methods used might not be as effective.Of course, further research is needed to attest to the inadequacies behind not attaining this goal as the scope of this research falls short of giving explanations other than mere speculations.