Lexical Evaluation of Teacher-made Coursebooks : Thai Case Studies of Foundation English Courses at Tertiary Level

Coursebooks as one of the major issues of language teaching / learning instruction can relieve the overburden as well as under-prepared teacher of a great deal of stress, time and additional work. Currently, both commercial and teacher-made English language teaching (ELT) coursebooks are available for use and seem flexible at hands. In any respect, however, usefulness of the coursebooks needs to be evaluated as claimed. To this end, the study, proposed as a new approach to lexical evaluation, examines the words occurring in three teacher-made ELT coursebooks used for compulsory foundation English courses in a Thai university as lexical syllabi are required to prepare university students for academic study at tertiary level. The RANGE program was employed to analyze the wordlists of the three teacher-made ELT coursebooks and it was found that those ELT coursebooks provided insufficient vocabulary size of general service words and academic words which were crucial to academic study regardless of syntactic structures. In addition, the three ELT coursebooks were frustrated for learners to survive unassisted reading in and outside the class. Application and suggestions are given for concerns and pedagogical practices.


Introduction
It is known that lexical syllabuses were of interest, discussions and debates among scholars in the field after Lewis's lexical approach had been proposed (1993;1997).Besides, in terms of research, a number of arguments about lexicon have been addressed and put forward to a more practice which in turn brought solid evidence to support the requirement of lexical syllabuses pertaining to a considerable size of the most common words of the language as well as patterns of word use (Sinclair and Renouf, 1988;Willis, 1990).Currently, such research findings have made the change in many ELT coursebooks in which coursebook writers must be more selective and principled about what vocabulary they should include (O'Dell, 1997).
Regarding learning English as a second or foreign language, it is generally accepted that on a systematic basis learners have to set themselves to acquire a significant portion of lexicon.Evidence from a number of studies in corpus linguistics shows that much of language includes a high proportion of such prefabricated language as it is actually used in speech and writing.According to Sinclair (1991), formulae is required for more of the choices which speakers and writers make than do novel constructions in relation to the application of grammatical rules.In addition, for both receptive and productive purposes, it is argued that foreign language learners have to improve a good L2 vocabulary so that they can make sense of what they hear and read and these enable learners to express themselves fluently and appropriately (Laufer, 1997).On the same issue, this is also addressed by Wilkins' advocate that "without grammar, very little can be conveyed.Without vocabulary, nothing can be conveyed " (1972: 111).
Although written texts, such as coursebooks and story books, contain thousands of running words, these running words (tokens) are made up of a much smaller number of different word types.If these word types are grouped into word families in the text, the number of items becomes less.This is because one family may include several types.Recognition of the relationship between vocabulary size and readability has its roots in early attempts to compile lists of the most frequently used words to be taught to language learners.Well known examples of word lists are Thorndike and Lorge's (1944) list of the 30,000 commonest words in English, West's (1953) 2,000 General Service List of English Words, and Hindmarsh's Cambridge English Lexicon (1980).A practical application of word lists is for the introduction of words into the vocabularies of learners in the order of their relative commonness, and as a guide to control linguistic input, particularly in written material.
The vocabulary items in a text can be grouped into the categories of high frequency, academic, technical, and low frequency words (Nation, 2001: 7).High frequency words are those which occur frequently in all kinds of texts.The words, identified in frequency counts across various types of texts, consist of about 2000 word families, as in the General Service List (West, 1953).Jenkins (1993), Sutarsyah (1993), Hirsh (1992) and Hwang (1989) studied the relationship between words and readability in various types of texts: reading texts for beginning ESL students, economics and academic texts, short novels, and newspaper articles respectively.They found the 2000 most frequent words accounted for around 83% of the text coverage.This means that if a text consists of 100 words, 83 of the words belong to the set of the 2000 most frequent words.This percentage changes with different kinds of texts, but is generally in the 80% to 90% coverage range.
It is evident that knowing these 2000 words alone is not adequate for reading, even though they account for the majority of the words in a text.Nonetheless, it is not necessary to know all the words in a text to comprehend it.A range of studies suggests that around 95% text coverage is adequate to comprehend a text (e.g.Ostyn and Godin, 1985).Laufer and Sim (1985) suggest that 95% text coverage is the minimum for adequate comprehension of texts, and should thus serve as a lexical threshold for texts.Laufer (1987) also points out that to achieve this coverage, a vocabulary of 5000 words should be learnt.Similarly, Sutarsyah et al. (1994) calculated that to read an economics text, a learner of English should have a vocabulary of at least 4000 to 5000 words.On the other hand, Hazenberg and Hulstijn (1996) argue that figures of 3000 or 5000 words are underestimates and that, at least for non-native speakers of Dutch at universities in the Netherlands, a vocabulary of 10,000 Dutch words is a more realistic minimum size for understanding first-year reading materials.As having an adequate vocabulary size is critical for reading ability and reading comprehension, Nation (1990: 24) states that learners of English as a foreign language need a productive knowledge of at least 3000 high frequency English words in order to be able to cope with university reading tasks.Nation (2006) also continues his argument on the same issue that taking 98% as the ideal coverage, a 8000-9000 word family vocabulary is needed for dealing with written text, and 6000-7000 families for dealing with spoken text.Thus, whatever the vocabulary size is, it is recommended that learners should know most of the words needed to make up the coverage for readability and understanding of text (West, 1914).He further points out that 98% of the words in a text should be known.That means that there will be one unknown word in every five lines.This is in line with what Hu and Nation (2000) found in a study of unknown vocabulary density and reading comprehension: to gain adequate comprehension from unassisted reading, at least 98% coverage of the text is required.Laufer (1987Laufer ( , 1992) ) found evidence of a strong relationship between measures of learner vocabulary size and text comprehension.
Given these facts -the broadly recognized need to focus on lexis in language learning, and it seems logical to reconsider the primary source of lexis like a coursebook as several additional factors are claimed if learners are targeted to be independent readers at a tertiary education level: 1) The classroom is unlikely to be able to provide all the lexis learners will need to survive their academic knowledge as independent readers can do (Willis, 1990;Lewis, 1993;Sokmen, 1997).
2) Lexical needs are largely unique to the individual, based in part on personal preference, professional and academic particularities, etc. (Rivers 1983;Nation, 2001).
3) According to modern technology, opportunities exist outside the classroom for developing one's lexicon.
Considering such mentioned factors, it can be argued that syllabus designers, coursebook writers and classroom teachers are not only justified but even duty-bound to devote some attention to a vocabulary portion required for learners' academic improvement at a university level.
For Thai university students as well as well-designed materials used in learning and teaching English at a tertiary level, the question like "how many words does a university student in Thailand need to know?" seems questionable and needs to be investigated.And there are several possible ways to answer this question.One is to look at what the students themselves aspire to.In Thailand, as in numerous other countries where English is a foreign language, university students are taught through the medium of the national language and they don't have much opportunity to engage in other languages than their native one.However, some of them who major in English language have expressed hopes of reaching native-like levels of proficiency.The other is readability which relates to a vocabulary size readers bring to their reading.
In terms of reading, meeting new words in written text is the main way new vocabulary is acquired.Many scholars in the field of reading (e.g., Nagy, Herman & Anderson, 1985) argue that reading is critical to second language vocabulary and thus if learners want to enlarge their L2 vocabulary size, they clearly need to be able to read successfully.However, finding learners suitable reading material is not easy and should be done with an understanding of text and its readability.According to Clay (1991), for instructional purposes, the text and its readability relate to how much the text matches learners' reading ability.A text in which learners can read 90%-95% of words easily is considered to be at learners' instructional reading level, that is, the level where instruction will be most effective.A text that is easier is considered to be at the learners' independent reading level.A more difficult text is considered to be at the learners' frustration level, and is not appropriate for reading instruction.To match text and learners, the level of difficulty of the text and the learners' reading ability need to be identified.Moreover, in terms of word learning, teaching a word does not mean that learners can learn it.According to Nation (1990: 41), we do not learn a word from one meeting.Research tells us that it takes between 5-16 meetings (or more) to 'learn' an average word.
With regard to vocabulary and readability, the present study dealing with a coursebook evaluation was conducted based on a lexical framework used as the main criteria as follows: 1) Core vocabulary for ES/FL learners as GSL (West, 1953) and AWL (Coxhead, 2000) lists; 2) Vocabulary and Readability as text coverage of 95% (Laufer, 1992) and 98% (Hu & Nation, 2000); 3) Recycling of word for word learning as word repetition (Nation, 1990(Nation, , 2001) and text for instructional purposes (Clay, 1991).
The primary objective of the present study, then, is to investigate the English vocabulary size occurring in the ELT coursebooks used for three compulsory foundation English courses for Thai university students in order to find out any overlap of words among the three texts.The second objective is to investigate whether there have been a substantial number of English words at various levels which provide students with enough portions of words required for their academic study in a university as well as for doing independent reading at the university level.In addition, in-house texts or coursebooks and language placement and proficiency tests are expected to be revised and developed according to the data emerging from this study.To fulfill the objectives of the study, the following research questions are answered: 1) How many high frequency and academic word families are provided in the three teacher-made ELT coursesbooks for teaching and learning?
2) Can the three teacher-made ELT coursebooks prepare university students for academic and independent study at a university level?

Methodology
The present study involved comparing the words appearing the three ELT coursebooks.The major source of data for the study consisted of the words actually used in the three ELT coursebooks classified by the RANGE program.

The Computer Program
The Range program is a Windows-based program developed by Paul Nation and Alex Heatley (2002) of the Victoria University of Wellington and is freely downloadable (see more detail in Wan-a-rom, 2008).It can be used with three distinct word lists, called baseword lists, on any text.The baseword lists contain word families.For example, the headword ABLE is grouped with its family members abler, ablest, and ably.Thus, the three family members are counted as the same word, ABLE.The Range program can sort a text's vocabulary into three categories of word families from each list and a category of words outside all the three lists, making four categories altogether.The program can do this either by range across several texts or by frequency within a text.It can also mark each word according to the category in which it falls.The baseword lists can be altered depending on specific requirements.The ones that come with the program are the first and second thousand words from the GSL (West, 1953) and Averil Coxhead's (2000) Academic Word List (AWL).The program has self-checking routines to ensure that a word form does not occur in more than one of the baseword lists.This program has been used with the text-based studies of Hirsh and Nation (1992), Laufer and Nation (1995), Coxhead (2000), Chung and Nation (2004), Nation (2006), and Wan-a-rom (2008).

Three Teacher-made ELT Coursebooks as Targeted Texts
The three ELT coursebooks chosen as case studies are Preparatory English (PE), Foundation English 1 (FE1) and Foundation English 2 (FE2) coursebooks which were written by university lectures in the Department of Western Languages and Linguistics in one of the Thai government universities in north-eastern Thailand.The three ELT coursebooks were used as medium for language taught to prepare university students for being academic and independent readers who can survive English language as a tool for academic study at a university in and outside the class.The first course -Preparatory English course -was designed to review the language taught at high school level while the other two courses introduced essential English the university students should know.Each coursebook was designed in themes ranging from what surrounds the students to what happens in the world in order to meet the requirements of General Education in a university.Therefore, words and syntactic structures taken into an account of teaching and learning were likely to be various depending on the themes.However, the three courses were believed to work out in the way of instructional materials which can be adapted for various teaching conditions in terms of a large-size class.

Procedure
Because the study involved comparing words in the wordlists of the three coursebooks, the three original wordlists from each coursebook, which are in lemmas, had to contain the same kinds of word families.To obtain good matches between the word families in the lists of the three coursebooks, a standard set of word families had to be made.
Step I: Preparation of scanned texts and GSL and AWL basewordlists.The words the three coursebooks were scanned and checked against the original coursebooks to avoid mistakes and errors.Then the texts were put in a text file.GSL and AWL basewordlists in the RANGE program were checked against the GSL and AWL lists.
Step II: Construction of the baseword lists.In this study, a word family is defined according to the idea put forward by Bauer and Nation (1993).A word family consists of a baseword and all the derived and inflected forms that can be understood by a learner without having to learn each form separately. Bauer and Nation used frequency, productivity, and regularity as the criteria for establishing the various levels of a word family.Level 3 of the Bauer and Nation scheme was used because this includes all the inflected forms and a small group of high-frequency, regular, and productive derived forms.This level seemed most suitable for the proficiency of the learners who would be reading the graded readers.
The inflectional categories are plural, third person singular present tense, past tense, past participle, -ing, comparative, superlative, and possessive.
The derivational affixes allowed at Level 3 are -able, -er, -ish, -less, -ly, -ness, -th, -y, non-, and un-, all with restricted uses.The wordlists of the three coursebooks were modified according to the following criteria: (a) The same words in both lists must have the same family members; (b) a family member in one list cannot be a headword in another list; and (c) a compound word in both lists is treated in a similar manner, that is, a hyphen is taken out to let the basewords stand alone or the word is used without a space or a hyphen in both lists.
A major weakness of the Range program is that it deals with word forms.Thus, it was not able to distinguish words' parts of speech and meanings, namely, words that had the same written forms but different meanings; for example, march (n.) and march (v.) were recognized as the same word by the program.This problem also occurred with most words that do not change their written forms to indicate tense such as put and shut.However, the latter problem does not matter much because whether such verbs are in present or past, they do not change meaning and are members of the same families.The same problems were found in both the wordlists compared.
Step III: Comparison of the wordlists.Before the wordlists of the three coursebooks were compared, the baseword lists of the three coursebooks were carefully checked to make sure that all the words in the three coursebooks were included and that they all had the same family members: 1).The baseword lists of the three coursebooks were run against the lists to check accurate matching of the headwords and the words in the coursebook text files.
2).The baseword lists of each pair of two coursebooks (Preparatory English and Foundation English1, Preparatory English and Foundation English 2, and Foundation English 1 and Foundation English 2) was then combined, and that combined list was used to make sure that the same family members were included under the same headwords.This was to check that all family members under the same headwords in the three sets of baseword lists were the same.
3).The three new baseword lists for each coursebook were constructed from the combined lists.Reestablishing the three new baseword lists after rechecking all headwords and their family members in the combined lists avoided some errors that might happen with some headwords in either lists.Figure 1 illustrates these procedures.
Figure 1.A flowchart of how three comparable baseword lists for the two coursebooks were constructed

Results and Discussion
Answering the main research questions involved looking at the number and overlap of words in the wordlists of the three teacher-made ELT coursebooks.There are several ways of comparing the lists.We will look at each of them in turn.They are:

Comparing the Number of Words Appearing in Terms of Word Families and Coverage against GSL and AWL
The figures in Table 1 show four things about the three wordlists: (1) the number of 1 st 1000 word families, (2) the number of 2 nd 1000 word families, (3) the number of academic word families, and (4) text coverage in each coursebook.As we can see in Table 1, there are five aspects to consider among the three teacher-made ELT coursebooks.They are the number of GSL, AWL and other words, and text coverage.It is obvious that as a whole, according to the growing number of words of each kind in each row, the three teacher-made coursebooks gradually provide words for students to learn when they move up to the next course.When the number of GSL and AWL used as criteria for words necessary for learners to know and learn, all the three teacher-made ELT coursebooks could provide a considerable number of 1 st 1000 words -753, 785, and 810 out of 1000 words respectively.It is good for learners to make use of those words as basic words for better understanding of text.In terms of 1 st 1000 words which are required to be basic words for readingability, Foundation English 2 does a better job than the other two do.However, the three teacher-made coursebooks do not seem to be good for learners who still need to know more about some basic words like the 2 nd 1000 words since they cannot provide enough words to be used as useful context for reading.Based on small portions of those words introduced in each coursebook (34%, 37%, and 35%), learners might find it hard to read each coursebook independently.Similarly, if learners need a bigger size of AWL words for their academic study, the three teacher-made coursebooks are unlikely to be successful in this point since they can provide 35% of words of this kind on average while the number of other words which consist of proper names and low frequency words is getting higher (762, 1131, and 1010 words) as found in the three teacher-made coursebooks.Such numbers affect the readability and reduce percentage portions of text coverage.That is, in terms of readability, Preparatory English cousrebook is likely to be the easiest according to text coverage of 72.60% while the other two -Foundation English 1 and Foundation English 2, provide 68.48% and 70.71 % respectively.However, all are lower than preferable coverage.On the same issue, among the three teacher-made ELT coursebooks identified by the number of GSL and AWL words, Foundation English 1 is the most difficult.Thus, as 95%-98% coverage is needed for independent reading, each of the three teacher-made ELT coursebooks seems hard for learners to survive their own independent reading and none of these coursebooks is appropriate for word learning when assistance is not given.

Comparing Word Repetition in Each Wordlist of the Three Teacher-made ELT Coursebooks against GSL and AWL
To look at word repetition, how frequent each word appears in the teacher-made ELT coursebooks is focused.With a good routine of the RANGE program, it is easy to find how frequent each word appears in a text because this is automatically done by the computer program designed in the RANGE program.To this regard, how many words appear frequently enough for word learning is considered according to a criterion of good word repetition (Nation, 2001).He argues that 5-16 meetings or more are good for word learning.So, in the present study, at least 5 appearances of word in the entire text is account for a good appearance as shown in Table 2.In Table 2 we can see that the number of appearances of the words for the frequent 2000 words is quite similar.74 words out of 753 1 st 1000 words in Preparatory English coursebook appear at least 5 times which make 10% while 6 words out of 338 2 nd 1000 words in the same text appear at least 5 times giving only 2%.The same pattern of appearances happens with Foundation English 1 and 2 coursebooks which give very small portions of word repetition of each kind of word.In general, the three teacher-made ELT coursebooks do not seem to be a good source of word learning in terms of word repetition.In other words, this means that learners do not have much opportunity to meet the same words when working through the whole book.

Comparing the Total Overlap of the Words in the Wordlists of the Three Teacher-made ELT Coursebooks against GSL and AWL
The overlap between the two wordlists consists of the word families which occur in both wordlists.If the three wordlists are compared, the overlap among the three wordlists refers to the words occurring in the three wordlists.In other words, the three wordlists share the same words.
As we can see in Figure 2, there are 666 1 st 1000 words occur in the three teacher-made coursebooks.This makes considerable proportion of the words three teacher-made coursebooks share out of 1 st 1000 words (66%).However, the proportions of the 1 st 1000 words include in the two wordlists are not high: 14% in PE and FE1, 18% in PE and FE 2, and 25 % in FE 1 and FE 2. In terms of recycling of words that enhances word learning, learners are likely to meet rather big size of the same 1 st 1000 words at reasonable degree.
When the overlap of 2 nd 1000 words are looked at, there is a small proportion of words occurring among the three teacher-made coursebooks if compared to the proportion of 1 st 1000 words occurring in the three texts.Figure 3 illustrates that 204 2 nd 1000 words are shared by PE, FE1 and FE2.This figure makes only 20% of the just met words which learners have opportunity to review when doing some reading through the three texts.In a similar manner, the number of 2 nd 1000 words reoccurring between the two texts is not large.That is, there are 54 2 nd 1000 words (54%) in PE and FE1 while 46 2 nd 1000 words (46%) in PE and FE2, and 75 2 nd 1000 words (75%) in both FE1 and FE2.Such proportions of the overlap of 2 nd 1000 words show that learners would find hard to learn the new words in the texts since inadequate vocabulary size of 2 nd 1000 words is provided.The number of such words also shed light on the idea that learners might find it difficult to guess the meaning of word from context when reading such texts.
Figure 4 points out the small number of overlap of AWL words among the three teacher-made ELT coursebooks.Only 62 AWL words which are very important for academic study at a university level reoccur in the three texts.This figure makes a very small proportion of word recycle, that is, 11% (62 divided by 570).The similar pattern can be found in PE and FE1, PE and FE 2, and FE1 and FE2 which give 0.06 %, 0.05%, and 0.14% of word recycle respectively.When the overlap of the three texts is considered, in terms of learning words, it can be said that 11% of recycling of AWL words does not make any difference for learners in gaining a substantial number of words required for academic study at a university.3.With regard to comparison of the three coursebooks using the three criteria in the present study, it was found that: 1).The three coursebooks, namely, Preparatory English, Foundation English 1 and Foundation English 2 contained 1 st 1000 words the most, that is, 75%, 79%, and 81% respectively.The proportions of 2 nd 1000 words and AWL words were not satisfied in terms of the words necessary for guessing and academic study, for example, 34% of 2 nd 1000 words in Preparatory English, 37% of 2 nd 1000 words in Foundation English 1, and 35% of 2 nd 1000 words in Foundation English 2 while AWL found in Preparatory English, Foundation English 1 and Foundation English 2 are 27%, 38%, and 40 % respectively.To this end, if 95% coverage (Laufer, 1992) and 98% coverage (Hu & Nation, 2000) are applied for effective reading, the three coursebooks cannot reach the aim of independent reading.According to inadequate numbers of GSL words required for constructing a good context for reading, learners might find it hard to survive their own reading at their own pace.The coverage found in the three cousrsebooks (72.60% for Preparatory English, 68.48 % for Foundation English 1, and 70.71 % for Foundation English 2) points out that none of them can be a good source of reading with particular reference to density of the unknown words indicated by small coverage which is below preferable coverage as advocated by Laufer (1992) and Hu and Nation (2000).
2).According to inadequate numbers of word repetition which was shown by proportions of frequencies of words in the coursebook itself or across the three coursebooks.The small proportions of word repetition (Nation, 2001) such as 10% in Preparatory English, 12% in Foundation English 1, and 11% in Foundation English 2 could be evidence that incidental learning of most words s unlikely to occur.Regarding proportions of word repetition or recycle (Nation, 2001) across the three coursebooks: 66% for 1 st 1000 words, 20% for 2 nd 1000 words, and 11% for AWL words, this thus would be not good enough to prepare learners for academic and independent study at a university level.
3).Overall, the three coursebooks are unlikely to enhance word learning strategies and independent reading due to text coverage which is lower than the ones required for guessing from context, that is, 95% (Laufer, 1992), or even for gaining good comprehension which relies on 98% (Hu & Nation, 2000) when unassisted reading is done.
4).Based on the three criteria, the three coursebooks seem frustrated for learners as independent readers to struggle for good comprehension and vocabulary learning as suggested by Clay (1991).This might also affect teaching without well-prepared exercises and well-designed activities.

Application to Pedagogy
According to the study results, some findings should be taken into consideration so that the three coursebooks could benefit learners both in and outside the class.
1).According to inadequate proportions of basic vocabulary required for independent reading, revision of the GSL particularly 2 nd 1000 words should be made to provide sizable portions for vocabulary teaching and learning for each coursebook.This can be done through the baseword lists of the three coursebooks.Also, such baseword lists can be used as guidelines for elaborating low frequency words (most difficult words) in some reading context for better reading.
2).Based on the density of the AWL words introduced with an insufficient portion of the GSL, explicit instruction and well-designed tasks are required for more possible incidental learning of words.Such tasks might enhance recycling words which learners can take as wrap-up exercises to conceptualize the words for word retention after class.
3).As by-products of the study, adjusted (established) wordlists for each of the three coursebook could be used as a basis for designing General English (GE), English for Academic Purposes (EAP), and English for Specific Purpose (ESP) coursebooks at a higher level in a university as well as designing in-house tests such as a placement test and a proficiency test for each course.

Suggestions for Further Study
Although the present study seems to work on a narrow aspect like a lexical perspective, a new approach to lexical evaluation of ELT coursebooks using the RANGE program as a corpus study should be proposed and practiced as suggested: 1).Replication of the present study could be done with other ELT coursebooks both commercial and teacher-made to see whether those coursebooks have provided learners with sufficient words for academic study and independent reading at any levels.
2).Research on evaluation of the established wordlists and word learning should be conducted as a follow-up study to see whether such wordlists work out and actually enhance incidental learning of words as assumed.
3).With reference to the established wordlists derived from the study, research on in-house placement tests as well as other proficiency tests should be carried out for an in-house test bank.
Previously, evaluation of ELT coursebooks has never been done from a lexical perspective because there is a problem with constructing a computer program for checking words used in the coursebooks.The present study is done with a narrow focus on lexis as a preliminary case study although there might be some limitations in terms of syntactic structures.Above all, coursebook writers particularly teachers or the persons in academic circles should be aware of words planned to be learned or even words used as supportive context for both successful unassisted reading and instructional purposes.From a lexical perspective the present study is intended to be a further step towards bettering the quality of teaching and learning vocabulary in a second or foreign language class at a tertiary level where teacher-made ELT coursebooks could be successfully exploited by both teachers and learners in and outside the class.
Abbreviations such as the following are located under their word families.ROAD: RD STREET: ST MOUNTAIN: MT FEBRUARY: FEB VOLUME: VOL

Table 1 .
Words appearing in the three teacher-made ELT coursebooks against GSL and AWL

Table 2 .
Word repetition in the three teacher-made ELT coursebooks

Table 3 .
Findings found in comparison of the three teacher-made ELT coursebooks