An Evaluation of English Language Grammar for College Students 1 & 2 : An EAP Coursebook Evaluation

The purpose of the present study is to do a detailed evaluation of English Language Grammar for Iranian College students. In order to accomplish the purpose of the research, the researchers used two instruments: 1) The questionnaire developed by Daoud and Celce-Murcia 2) The researcher-made checklist based on Bloom's taxonomy of language learning. Regarding the questionnaire, twenty available EFL teachers evaluated the book in terms of five key aspects of subject matter, vocabulary and structure, exercises, illustrations, and physical make up. Concerning the checklist, the researcher meticulously evaluated the book in terms of three domains of cognitive, affective, and psychomotor. In order to analyze the data in both instruments, descriptive as well as inferential statistics such as Kruskall-Wallis and Pearson Rank Order Correlation tests were applied. The result of statistics for the questionnaire revealed that subject matter, vocabulary and structure, and physical make up of the book are emphasized and two aspects of exercises and illustrations are ignored. And there is only correlation between subject matter and physical make up. Data analysis for the checklist indicated that within cognitive domain, only first stages are emphasized and the last stages of synthesis and evaluation are neglected. In addition, cognitive domain is more privileged than affective and psychomotor domains. Finally, some pedagogical implications for both teachers and syllabus designers are provided.


Introduction
With the growing shortage of time and money for writing textbooks, there is a premium on making effective use of what already exists.Stevick (1971) stated that, for any given set of materials the choice was only between using them and rejecting them.Adaptation, as a third alternative, has received very little either of time or of money or of prestige.Rewriting, a fourth possibility, is often viewed both as unjustifiably troublesome for the rewriter, and as an affront to the original author.Chambers (1997) considered evaluating materials as a complex process.First, it demands that teachers establish their relative merits from among a wide range of features.Pedagogical factors to be considered include suitability for the age group, cultural appropriateness, methodology, level quality, number and type of exercises, skills, teacher's book, variety, pace, personal involvement, and problem solving.Second, we have to bear in mind not only construct validity or the extent to which a reviewer thinks that a book will or will not be useful to a specified audience, but also the materials already in use.Third, we need to consider whose views we wish to consider in exercise.Richards (2001) believes that before one can evaluate a textbook, information is needed on the following issues: -The role of the textbook in the program.
-The teachers in the program.
-The learners in the program 2. Theoretical Framework Materials evaluation is initially a time-consuming and difficult undertaking.Tomlinson (2003) stated that doing evaluations formally and rigorously can eventually contribute to the development of an ability to conduct principled informal evaluations quickly and effectively when the occasion demands (e.g., when asked for an opinion of a new book; when deciding which materials to buy in a bookshop; when editing other peoples' materials).As Richards (2001) notes, teachers and others responsible for selecting materials have to choose from such an array of existing textbooks.This choice should be based on informed judgment about the teaching materials.However, as Robinson (1991) asserts, since no textbook is perfect, choosing a right textbook to meet the specific needs of learners is a very difficult process.Sheldon (1988) states that "the selection of a coursebook signals an executive educational decision in which there is considerable professional, financial and even political investment" (p.237).This is where textbook evaluation comes into play.Evaluation is required not only in situations where the teachers are responsible for selecting the textbook, but also in systems in which the school board or the state has the responsibility of adopting textbooks.In either case, teachers need to be familiar with the evaluation process in order to apply its assets and compensate for the problems and weaknesses of an adopted textbook to make the author's biases more harmonious to the needs of the specific students who are going to use the book (Skierso, 1991).However, Williams (1983, cited in Ansary & Babaii, 2002) claims that "it is ironical that those teachers who rely most heavily on the textbooks are the ones least qualified to interpret its intentions or evaluate its content and method" (p.1).Thus, it is vitally essential to provide teachers with some information about how to recognize a good textbook, what the main criteria for textbook assessment are, and what checklists or schemes are available to be used.Hutchinson and Waters (1987) believe that even if a teacher finally decides to write his own material, the evaluation of ready-made materials can give him lots of insights of what to do or what to avoid.Besides, by revealing existing materials which meet all or part of his materials needs, the evaluation can save lots of duplication effort.This argument is in line with Ellis' (1997) statement about the role of textbook evaluation.For him, textbook evaluation can be very useful in teacher development and professional growth.Textbook evaluation helps teachers move beyond impressionistic assessments and it helps them to acquire useful, accurate, systematic, and contextual insights into the overall nature of textbook material.In other words, textbook evaluation can be a worthwhile tool for conducting action research as well as a form of professional empowerment and improvement.According to Johnson (1982), teaching and learning materials provide the corpus of the curriculum.They normally exist as physical entities and are open to analysis, evaluation and revision in ways that teaching and learning acts are not; and they have a direct influence upon what happens in classrooms, which policy documents syllabuses and teacher-training courses do not.Yen (2011) conducted a case study of the foreign cultures represented in two English language textbooks used by Hong Kong secondary schools.Its aim was to investigate whether the representation of foreign cultures in these textbooks reflected the status of English as an international language.In order to do this, references to foreign cultures were categorized into four aspects: products, practices, perspectives, and persons.It was found overall that the representation favored the cultures of English-speaking countries, while the cultures of Africa were underrepresented.Recently, Amiryousefi and Ketabi (2011) aimed to first summarize some of the arguments put forward by the pro and anti-textbook camps and then discussed the results of a complex evaluation process of the EAP textbooks used at the four leading universities of Iran to assess whether these anti-textbook ideas had any validity in the field of EAP specially in an Iranian setting.The evaluation was done via three questionnaires (namely Students Needs Analysis Questionnaire, Students Textbook Evaluation Questionnaire and Teacher Textbook Evaluation Questionnaire) answered by more than 300 EAP students and teachers.In combining the results of the three questionnaires used, it was found that although a lack of fit between the needs of the students and the textbook contents and organizations was rather apparent, EAP textbooks deemed necessary and useful for EAP classes.Teachers should, therefore, be given more autonomy to accommodate students' needs and interests when essential.

Purpose of the Study
In spite of the counter-arguments made against using a textbook, it is still the most widely exploited material by many educators, and as Sheldon (1988) puts it, they are frequently viewed as "necessary evils" (p.237).It is not used just by educators; even policy makers and administrators rely heavily on textbooks to achieve prescribed goals and objectives.Therefore, it is vital that all involved individuals at all levels of a program consider the importance of their decisions and try to effectively match textbooks with the identified needs of students (Garinger, 2002).Riaz ( 2003) stated the fact that there are a vast number of published materials for English language teaching available in the markets which makes selecting the right course book a challenging task.Therefore, evaluation task seems logical and plausible.In this regard, English language teachers and administrator need to get acquainted with the principles of textbook evaluation as well as the available evaluation checklists.The purpose of the present study is to evaluate English Grammar for College Students 1 and 2. This book is intended for the students majoring in the English language in the universities and institutes of higher learning across the country.Since the book is intended for an English grammar course of eight credits, to be covered during two semesters, the researchers tried to both evaluate the usefulness of the book and determine whether the book is well worth studying and teaching.Given such theoretical underpinnings, this study sought to explore the following research questions.Descriptive questions for questionnaire one (five aspects of the book) 1).How suitable is the subject matter of the book?2).How suitable is the vocabulary and structure of the book?3).How suitable is the exercises of the book?4).How suitable is the illustrations of the book?5).How suitable is the physical make up of the book?Inferential questions for questionnaire one 6).Are there any correlations among five aspects within checklist 1? Descriptive questions for checklist one 7).Which levels within the cognitive domain are tapped?8).Which domain is more emphasized?Inferential questions for checklist one 9).Are there any significant differences among different levels within cognitive domain?10).Are there any significant differences among three domains of cognitive, affective, and psychomotor?11).Are there any correlations among the three domains of cognitive, affective, and psychomotor?4. Method 4.1 Participants Participants included in this study were twenty EFL university lecturers in Islamic Azad University.The sample group (n=20) was all the university teachers which were on hand.

Instrumentation
In order to accomplish the purpose of the research, the following three instruments were used.

Textbook
The researcher evaluated English Grammar for College Students 1 and 2 (Azabdaftari, 2003).This book is intended for the students majoring in the English language in the universities and institutes of higher learning across the country.4.2.2Questionnaire and Checklist -Questionnaire (Textbook evaluation questionnaire prepared by by Daoud and Celce-Murcia (1978)).This questionnaire was prepared and distributed to twenty EFL lecturers.It included five different aspects of subject matter, vocabulary and structure, exercises, illustrations, and physical make up of the book.Each of the major sections was divided into different subsections (4, 9, 5, 3, and 4 respectively) and the raters chose a point along a scale (5-point Likert scale).
-Checklist (Textbook evaluation checklist prepared based on Bloom et al. (1956) taxonomy of education objectives and modified by Hashemnezhad (2010).This checklist included three cognitive, affective, and psychomotor domains.Each of the three major sections was divided into different subsections.Cognitive domain included six subsections of knowledge, comprehension, application, analysis, synthesis, and evaluation with ten Yes-No questions for each.There is ten Yes-No questions for each of affective and psychomotor domains.

Procedures
As it was mentioned before, two research instruments of questionnaire and checklist were used in the analysis.The first one was Dauod and Celce-Murcia's (1978) questionnaire which evaluates the book in terms of five different aspects of subject matter, vocabulary and structure, exercises of the book, illustrations of the book, and finally physical make up of the book.The questionnaire was given to twenty university teachers of English who had enough experience in teaching the book.The questionnaires were returned and the answers were analyzed.The second instrument was a checklist prepared by researcher himself based on Bloom et al. ( 1956) taxonomy.There are three main domains of cognitive, affective, and psychomotor domains with some levels.The researchers analyzed the whole book in terms of three main domains.

Conclusion and Interpretation of Results
In terms of the first five descriptive questions, as Table (1) and Figure (1) illustrate, the suitability of the analyzed book varies within the following range: Subject matter (2.55), vocabulary and structure (2.50), physical make up (2.26), exercises (1.78), and illustrations (1.50).Regarding the inferential question for questionnaire one (question 6), as Table (2) shows, there is only correlation between Subject matter and Physical make up.Correlation coefficient between the two variables at the 0.01 level is (-1) and P value is (sig=0.000)< 0.01.Concerning the descriptive questions for checklist one (questions 7 and 8), Table (3) and Figure (2) show the mean scores for five levels within cognitive level as follows: Application (14.50), knowledge (7.90), analysis (3.70), comprehension (3.10), synthesis (2.90), and evaluation (1.90) respectively.
In order to answer question number 8, Table (4) and Figure (3) illustrate that cognitive domain is more emphasized and the mean scores for three domains are as follow: Cognitive domain (56.67), psychomotor domain (48.00), and Affective domain (16.00).In terms of inferential questions for checklist one (question number 9), the result of Kruskall-Wallis test (Table 5) shows that chi-square at significant level of (0.05) with df=5 is (chi-square=9.596)and since P value (P=0.088)> 0.05, we conclude that there are no significant differences among the different levels within cognitive domain.In terms of inferential question for checklist one (question number 10), the result of Kruscall-Wallis test (Table 8) shows that there are significant differences among the three domains of cognitive, affective, and psychomotor because chi-square at significant level of 0.05 with df=5 is 6.144 and P value (P=0.046)< 0.05).In terms of the last inferential question of checklist one (question number 11), the result of Table ( 9) illustrates that there is only correlation between affective and psychomotor domains because correlation coefficient at the 0.05 level is 0.646 and P value is 0.044 < 0.05.

Pedagogical Implications for Teachers and Syllabus Designers
Reform of education is not simply reform of school system but reform of the behavior and thinking of the wider social teaching-learning process that guides moral-political ideas and behavior.Far-reaching curriculum innovation involves fundamental shifts in the values and beliefs of the individuals concerned (Burns, 1996).Decisions related to textbook selection will affect teachers, students, and the overall classroom dynamic.It is probably one of the most important decisions facing EFL educators.The use of an evaluation procedure or checklist can lead to a more systematic and thorough examination of potential textbooks and to enhanced outcomes for learners, instructors, and administrators.Regarding the results of data analysis the following pedagogical implications and solutions are recommended.1).Regarding descriptive questions for questionnaire one (questions 1 to 5), less attention has been paid to exercises and illustrations of the book and the syllabus designers and the author of the book should take these two important aspects into consideration.2).Concerning the research question number 6, there is only correlation between subject matter and physical make up of the book and there is no correlation between the other aspects.
3).As we know, cognitive domain proposes the following order of sub-domains: Knowledge, comprehension, application, analysis, synthesis, and evaluation.In order to gain enough proficiency and communicative competence and be able to produce the language, the learners should reach the final aspects of cognitive domain such as analysis, synthesis, and evaluation.As the results of data analysis show, less attention is paid to the final stages of cognitive domain.The writer should adapt the book in order to take the final and productive aspects into consideration.4).Concerning questions number 8 to 11, the results of data analysis reveal that cognitive domain is more privileged than affective and psychomotor domains.That is, the importance of affective and psychomotor domains is ignored in the book.It is obvious that affective domain emphasizes some internalized aspects of language learning such as positive attitude and interest of language learners toward language learning which have been neglected in the book.Psychomotor domain focuses on some productive aspects of language learning, such as performance, doing, writing, harmonizing learning with performing, verbal and non-verbal feedback, etc.In order to prepare learners to a fully-fledged acquisition of language and reach the mastery level of learning, the writer of the book as well as the syllabus designers should try to make full use of all 3 domains of cognitive, affective, and psychomotor domains.

Concluding Remarks
Regarding the role of materials in the language learning process, Edge (1993) remarks that "our purpose is not to teach materials; our purpose is to teach students and to use materials in that process" (p.43).All said, we would like to conclude this article with a quotation from Allwright (1981): There is a limit to what teaching materials can be expected to do for us.The whole business of the management of language learning is far too complex to be satisfactorily catered for by a pre-packaged set of decisions embodied in teaching materials (p.9).This means, however perfect a textbook is, it is just a simple tool in the hands of teachers.We should not, therefore, expect to work miracles with it.What is more important than a textbook is what we, as teachers, can do with it.As Brown and Yule (1983)  In order to find the correlation among variables rather than within variables the researcher used Spearman rank order correlation.Table (2) indicated that there is only correlation between two variables of subject matter and physical make up.Correlation coefficient between these two variables at the 0.01 level is (-1) and P value is (sig=0.000)< 0.01.The result of Table (6) shows that chi-square at significant level of (0.05) with df=5 is (chi-square=9.596).We conclude that there are no significant differences among the different levels within cognitive domain (P value (P=0.088)> 0.05) The result of Table (8) shows that chi-square at significant level of (0.05) with df=5 is (chi-square=6.144).We conclude that there are significant differences among the 3 domains of cognitive, affective, and psychomotor (P value (P=0.046)< 0.05).The result of Table (9) illustrates that there is only correlation between affective and psychomotor domains because correlation coefficient at the 0.05 level is 0.646 and P value is 0.044 < 0.05.

Figure 3 .
Figure 3.A bar graph representing means for three domains of cognitive, affective, and psychomotor Table (4) and Figure (3) indicate the mean for three domains of cognitive (56.67), psychomotor 948.00) and affective (16.00) respectively.

Table 1 .
put it: It is, in principle, not possible to find materials which would interest everyone.It follows that the emphasis should be removed from attempting to provide intrinsically interesting materials, which we have just claimed is generally impossible, to doing interesting things with materials… these materials should be chosen, not so much on the basis of their own interest, but for what they can be used to do (p.83).Descriptive Statistics for Questionnaire

Table 2 .
Spearman Rank Order Correlation

Table 3 .
Descriptive Statistics for Only Cognitive Domain

Table 4 .
Descriptive Statistics for Three Domains of Cognitive, Affective, and Psychomotor

Table 5 .
Frequency and Mean Ranks of Six Levels within Cognitive Domain

Table 6 .
Analysis and Test Statistics of Kruskal-Wallis Test for Six Levels within Cognitive Domain

Table 7 .
Frequency and Mean Ranks for three Domains of Cognitive, Affective, and Psychomotor

Table ( 7
) indicates that cognitive domain is more privileged than psychomotor and affective domains.

Table 8 .
Analysis and Test Statistics of Kruskall-Wallis Test for Comparing the Means of 3 Domains

Table 9 .
Spearman Rank Order Correlations to Find the Correlation among Three Domains of Cognitive, Affective, and Psychomotor Correlation is significant at the 0.05 level (2-tailed).*.