An Instructional Application of the Multiple-Choice Cloze: A Case Study in the EFL Classroom

We explored using multiple-choice cloze (MCC) tests for classroom instruction. The practice of “testing leading teaching” is frequently criticized because it might distort the original teaching objectives. We do not primarily emphasize how to get high scores; instead, we show how to use testing techniques and teaching activities to provide feedback that energizes teaching methods and increases learning effectiveness. We analyzed MCC test-taking strategies, which include leading students to: 1) skim for the first and the last sentence in cloze passages; 2) read the whole cloze passage to grasp its general idea; 3) look for contextual clues; 4) orally express (“thinking out loud”) their reasons for choosing one MCC test item instead of another; and 5) conduct group discussions. Finally, 6) teachers guided the entire class, discussed contextual and situational clues, and provided feedback about student choices and reasons. The experimental design of this research primarily compared the performance between two groups: Experimental and Control. Differences in cloze scores between the two groups were significant, but differences in reading comprehension scores were not. After six 25-minute MCC test lessons, Experimental group students had better MCC test scores than did Control group students. Our findings supported our hypothesis that MCC instruction, even for a short time, would improve performance on a cloze test. We also discuss how to use MCC tests to teach strategies for answering MCC test items.


The Development of Cloze
The term "cloze," pronounced like the verb "close," was coined by Wilson Taylor (Taylor, 1953).His idea was derived from Gestalt psychology, one of the central principles of which is that humans have an instinct for inferring a whole from unfinished patterns and figures.
Immediately after the cloze test was developed, a respectable body of articles on cloze measured readability and assessed reading comprehension in first-language (L1) teaching.Since then, relatively few studies have experimentally used cloze for instruction (Jongsma, 1971(Jongsma, , 1980;;Schoenfeld, 1980).By the early 1970s, cloze was used in second-language (L2) teaching, both as a reading activity and as a test tool (Steinman, 2002;Wu, 1994).
When cloze was first developed, most of the studies were primarily on traditional blank-filling cloze tests.They usually tried to answer the following three questions: 1) Can cloze procedure be used to measure overall reading comprehension, or does it just test a discrete language point?2) Which is the better way of blank deletion, fixed-ratio or rational deletion?3) Which is the better scoring method, the exact word or the acceptable word(s)?Fixed-ratio deletion means the blanks are deleted every nth word at random; rational deletion means the blanks are selectively deleted based on some rational decision.
After 1980, a modified multiple-choice version of cloze procedure (MCC in the present study) was introduced (Bensoussan & Ramraz, 1983, 1984;Saeedi, 2016;Scholz & Scholz, 1981).In an MCC test, students are asked to choose the best answer rather than to fill in the blanks with words.The blanks usually do not appear in a fixed ratio.Rather, they are designed to randomly test contextual and linguistic points.Despite a multitude of studies on MCC tests used as measuring tools, only a few have focused on MCC tests as instructional tools.
These studies suggest that MCC tests are a promising instructional tool and that they should be further investigated.

Cloze Instruction and Test-Taking Strategies
In the earlier development of cloze, some studies addressed cloze instruction and test-taking strategies.Jongsma (1980) reviewed and synthesized the 1970s literature on using cloze as a teaching tool.He arranged them into eight sections, the 5th of which was "analysis of teaching procedure."He concluded: 1) "Cloze instruction is likely to be more effective with discussion than without it" (p.13); 2) "Cloze instruction which is carefully sequenced, for example in length and difficulty, and adjusted to the reading abilities of students, is more effective than the undifferentiated use of cloze exercises" (p.15); and 3) "The quality of a cloze instruction program is more important than its length" (p.16).Carr, Dewitz, & Patberg (1989) said that students have to "activate prior knowledge, look forward and backward for syntactic and semantic information, and draw plausible inferences" (p.381).This kind of training will allow students to grasp the strategies needed for filling in the cloze blanks.Bortnick & Lopardo (1973, p. 299) suggested the following cloze instructional procedure: 1) "Read through the entire cloze passage silently.2) Reread the cloze passage, writing in words to fit the blanks.3) Offer reasons for the choices for these blanks.4) Compare the choices with the original passage.5) Be prepared to discuss."Cameron, Linton, & Hunt (1987, p. 2) recommended peer discussion to increase linguistic flexibility.
Synthesizing the cloze instruction and test-taking strategies stated above, we developed the cloze instruction package provided in Section 2.3.1 below.

Cloze and Reading Comprehension
In the cloze literature, it is customary to link cloze with reading comprehension because cloze was first used to measure readability and reading comprehension.Williams, Ari, & Santamaria (2001) claimed that MCC was an "effective measure of both lower-level word reading skills and high-order comprehension processes" (p.228).Greene (2001) said that "appropriately designed cloze tests permit valid assessments of reading comprehension."Other authors, however, disagree.Ashby-Davis (1985, p. 589) warned that cloze "is invalid as a test of reading comprehension."Heitzman & Bloomer (1967, p. 222) claimed that "the use of non-overt reinforced Cloze procedure is not effective for producing an increase in reading comprehension."

The Importance of the Study
Good tests should be reliable and valid (Bachman, 1990;Oller, 1979).A reliable test has consistent scores in various settings, and a valid test measures what it claims to measure.Practicality, however, is no less important.Scholz & Scholz (1981, p. 3) said that "practicality views a test in terms of how easy or difficult that test is to create, administer, score, and interpret."Because MCC tests are easy to administer and to score, they are popular in testing batteries.
In most countries, secondary students are required to take a national entrance examination to enter college (Ahmadia & Jalilian, 2012;Sazegar, & Motallebzadeh, 2017;Williams, Ari, & Santamaria, 2011).In these examinations, MCC tests are integral.They are also included in some internationally famous English proficiency examinations for nonnative speakers of English: TOEFL, and ILTS etc. (Ajideh & Mozaffarzadeh, 2012) Consequently, because the examination results determine students' university matriculation and English proficiency certificates, students, teachers, and the general public are generally aware of MCC tests.In the current literature on cloze, however, MCC tests do not yet have the attention they deserve.There are no large-scale and systematic studies on them.

Motivation of the Study
We primarily investigated whether students' cloze performance and reading comprehension can be improved using only short-term classroom instruction.Bortnick & Loparo (1973, p. 297) said that "a major limitation of past attempts at using the cloze procedure instructionally has been the manner in which the procedure was used: little actual teaching was involved."They then suggested that "the cloze procedure be used in a teacher-directed learning activity" (p.297).This implies that for the cloze procedure to be effective, the teacher must take an active role in instruction.They also said that "the cloze procedure lends itself to instruction in the use of contextual cues as a reading strategy." In the cloze literature, there have been various suggestions on how to use MCC test techniques in class (Bortnick & Lopardo, 1973;Carr et al., 1989;Eze, 2015;Grant, 1979;Hale, Stansfield, Rock, Hicks, Butler, & Oller, 1988;Mostow, Huang, Jang, & Weinstein, 2017;Weaver, 1979).Cameron, Linton, & Hunt (1987) concluded that "as instructional devices, when used in conjunction with peer discussion, cloze passages increase linguistic flexibility."Nevertheless, three decades later, few cloze studies offer sufficient statistical data to persuade teachers and students that cloze tests should be used in class.Thus, we did this study.

Participant Characteristics
We randomly selected approximately 180 non-English majors in four mandatory freshmen EFL (English as a Foreign Language) classes at a university in Taiwan to participate in the study.Their majors were diverse: tourism, environmental engineering, cosmetic science, etc.Their first language was Chinese, and they were 18-20 years old.All had completed 6 years of 2-hour-per-week EFL classes at their secondary schools, and their educational backgrounds were not otherwise significantly different.Their English ability was low-to-intermediate, except for a few whose English ability was advanced.We randomly divided them into the Experimental group and the Control group.The cloze pretest scores between the two groups were not significantly different, which meant that the participants were at the same EFL level at baseline.Six weeks later, at the end of the study, only 161 students remained; the 19 dropout students did not finish the posttests, were absent from more than two classes during the six weeks, or both.

Materials
The passages used both in the pretest and in the posttest were taken from Taiwan's Testing Center for Technological and Vocational Education website (TCTE) (https://www.tcte.edu.tw/down_exam.php).Another website, the Automatic Readability Checker (http://www.readabilityformulas.com/free-readability-formula-tests.php),used to check the difficulty levels of the passages, indicated that their readability was between 9th and 12th grade, based on analyses by seven readability formulas.

Procedures
At the beginning of the study, all participants took a pretest that included 15 MCC items and ten reading comprehension questions.A six-week cloze instruction followed in the Experimental group, after which, all participants in both groups took a posttest that included 35 MCC items and 10 reading comprehension questions.We calculated and graded the pretests and posttests and used t tests to analyze the results.

Cloze Instructional Package
Two different teachers who used the same textbook conducted the four classes.Both teachers conducted one class in the Experimental group and one class in the Control group.There were two 50-minute classes each week.The only difference between the two groups' class content was that in the Experimental group there was one 25-minute cloze exercise per week, which left most of the class time for the standard instructional program.Because 2.5 hours of the class time was used for cloze instruction, teachers in the Experimental group had to shorten some lessons from the assigned class textbook.In the Control group, however, the teacher taught only the standard instructional program: basal reading, listening, and speaking materials in the textbook.To evaluate the effects of the cloze instruction, the teachers in the Experimental class told students: 1) to read the first and the last sentences in every cloze passage, each of which was kept intact; 2) then to read the whole cloze passage to grasp its general idea.By looking backward and forward in the text for syntactic, semantic, and connective clues, the students made plausible inferences to use as answers; 3) to choose the correct answers by seeking contextual clues within and beyond the cloze passage; 4) to orally express ("thinking out loud") their reasons for choosing their answers for MCC test items; and finally 5) to discuss the contextual clues in small groups.The teacher guided the entire class, explaining and discussing "contextual and situational clues that motivate items."The last activity is important because teacher feedback "prevents the activities from being overly frustrating and ultimately counterproductive" (Meyer & Tetrault, 1986, p. 414).
We hypothesized that these strategies and activities would train students to infer information from textual clues and to integrate textual information with their prior background knowledge and linguistic knowledge (Ajiden, Yaghoubi-Notash, & Khalili, 2017;Bortnick & Lopardo, 1973;Carr et al., 1989).

Research Design
We designed the study to answer these two questions: 1) Can students' performance on MCC tests be improved using cloze instructional activities in an EFL classroom setting?2) Can students' performance on reading comprehension tests be improved using cloze instructional activities in an EFL classroom setting?Thus, the two working hypotheses (H 1 and H 2 ) of this study are: H 1 : Between the Experimental and Control groups, there will be a significant difference in MCC test scores before and after the cloze instructional activities.
H 2 : Between the Experimental and Control groups, there will be a significant difference in reading comprehension test scores before and after the cloze instructional activities.
The independent variables of the experiment were the students in the Experimental and Control groups, and the dependent variables were the students' cloze test and reading comprehension test scores.

Results
When examining the results, we must depict the different difficulty levels of the pretest and the posttest passages.The pretest cloze scores of the Control group (CBC) had a mean of 47.29, but the posttest cloze scores of the Control group (CAC) had a mean of 36.78.In contrast, the pretest reading comprehension scores of the Control group (CBR) had a mean of 58.84, but the posttest reading comprehension scores of the Control group (CAR) had a mean of 48.49.The means of the difficulty levels of the passages were: pretest cloze = 9; posttest cloze = 10.4;pretest reading comprehension = 11.05;posttest reading comprehension = 11.65.The posttest items were not decided by the authors.Instead, after the 6-weeks' experimental instruction had ended, three of the authors' colleagues were asked to randomly chose 7 from around 70 passages in the item banks in the websites of the TCTE, one of the national entrance examination organizations for students in vocational secondary schools in Taiwan.We thought that by having our colleagues randomly choose the posttest items, the experimental results would be more objective.To answer the first research question, we used the separate-variance t test for the mean difference between pretest and posttest EAC-EBC and CAC-CBC cloze test scores: t = 3.701 (P = 0.0003) with 155 degrees of freedom, which is significant when significance is set at P < 0.01.This supports H 1 .
To answer the second research question, we also used the separate-variance t test for the mean difference between pretest and posttest EAR-EBR and CAR-CBR reading comprehension test scores: t = 0.014 (P = 0.9890) with 146 degrees of freedom, which is not significant when significance is set at P < 0.05.This does not support H 2 .

Findings
Our findings that support H 1 -between the Experimental and Control groups, there will be a significant difference in MCC test scores before and after using cloze instructional activities-are consistent with some other findings in the cloze literature.Weaver (1979, p. 632) reported that "when students fill in the blanks in a cloze exercise, they make use of semantic and syntactic context clues to determine the missing words; therefore, working with students on cloze exercises is a useful way to build their ability to use context clues in their reading." Carr et al. (1989, p. 385) reported that when instruction extends over several weeks, students appear to internalize the process of inferring answers.Grant (1979, p. 699) said that "a body of literature which suggests that cloze be used for teaching has accumulated."Hale et al. (1988, p. 3) said that "the multiple-choice version of the cloze test has been less thoroughly researched than the completion cloze test, but the research that has been done is sufficient to allow certain hypotheses to be drawn."Steinman (2002, p. 296) claimed that "the collaborating, negotiating, and think-alouds that occur during most group work would be beneficial during cloze activities as well." In contrast, our findings did not support H 2 -between the Experimental and Control groups, there will be a significant difference in reading comprehension scores before and after the experimental treatment.This might be inconsistent with some other findings reported in the cloze literature Ajideh & Mozaffarzadeh (2012).One reasonable explanation of this deviation is that only six weeks of instruction using methods to improve results on MCC tests was insufficient to improve EFL students' reading comprehension skills.

Limitations of the Study
Our study has some limitations.Only 25 minutes per week for six weeks (2.5 hours) was devoted to teaching MCC test strategies and activities.Expecting overall reading comprehension ability to significantly improve after such brief instruction is unrealistic because reading comprehension is too complicated to improve after only a few hours of cloze instruction.

Pedagogical Implications
Because the six-week cloze instruction required only 2.5 of the 36 teaching hours in the entire semester (18 weeks × 2 hours/week), it will neither take too much class time nor turn English classes into teaching to the test coaching sessions (Li, 2009).Additionally, the MCC test instructional strategies and activities in this study are student-centered rather than teacher-dominated.That should lead to more active, energetic, and efficient EFL classes, which makes class time used for cloze instruction worthwhile.

Recommendations for Future Research
To verify our findings and to allow for drawing stronger and more generally applicable conclusions, future studies should recruit more participants and spend more time (e.g., 18 instead of 2.5 hours in one semester) on cloze instruction.Moreover, a broader range of English proficiency levels, academic areas, and cultural backgrounds might yield findings that are more convincing.Furthermore, investigating a wider range of cloze item traits might also help.
Although our study showed no correlation between short-term classroom cloze instruction and reading comprehension ability, larger and longer studies might reach different conclusions.
We conclude that more research is necessary to verify our findings.

Table 1 .
Descriptive statistics SD: standard deviation; EAC: posttest cloze scores of Experimental group; EBC: pretest cloze scores of Experimental group; CAC: posttest cloze scores of Control group; CBC: pretest cloze scores of Control group; EAR: posttest reading comprehension scores of Experimental group; EBR: pretest reading comprehension scores of Experimental group; CAR: posttest reading comprehension scores of Control group; CBR: pretest reading comprehension scores of Control group; EAC-EBC: difference between pretest and posttest cloze scores in Experimental group; CAC-CBC: difference between pretest and posttest cloze scores in Control group; EAR-EBR: difference between pretest and posttest reading comprehension scores in Experimental group; CAR-CBR: difference between pretest and posttest reading comprehension scores in Control group.

Table 2 .
The raw scores of the pretest and the posttest